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March  2007  Final  Report 
Puget  Sound  Infectious  Disease  Tracking  System 
Award  Number:  W81XWH  -  04-1-0418 


Introduction: 

This  study  looked  at  the  operational  efforts  required  to  develop  advanced  tools  for  public  health  surveillance  and  disaster  management 
and  developed  evidence-based  knowledge  to  help  shape  these  efforts.  The  approach  was  to  rapidly  leverage  existing,  deployed 
computerized  public  health  data  collection  systems  in  the  Puget  Sound  region,  and  build  several  new  components  to  create  a 
laboratory  for  the  “field  study”  in  this  proposal.  The  project  built  a  system  of  sentinel  sites  that  assisted  us  to  answer  the  following 
categories  (themes)  of  questions:  feasibility  -  feasibility  of  automated  and  manual  reporting  under  a  variety  of  settings  and 
implementation  models;  relative  utility  -  relative  utility  of  the  data  gathering  through  these  techniques  (evaluating  each  technique 
alone  and  the  synergism  from  using  both  techniques  in  the  same  population  for  surveillance  and  detection);  technical  and  policy 
approaches  -  developed  technical  and  policy  approaches  to  cooperation  including  the  implication  of  bi-directional  data  exchange 
between  civilian  and  military  public  health  surveillance  systems. 
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Statement  of  Work:  Puget  Sound  Infections  Disease  Tracking  System/  renamed  by  workgroup  to  Outbreak 
Detection  Information  Network  (ODIN) 


Task  1  Status  -  Research  Accomplishments 

Task  1:  Community  Initiative  -  Use  a  community  based  Completed  Fall  2003:  Received  letters  of  support  from  Seattle 
approach  to  assist  in  gathering  requirements  for  data  collection  &  King  County  Public  Health,  Kitsap  County  Health  District, 
and  data  sharing  across  jurisdictions.  (Months  1-6)  Washington  State  Department  of  Health,  Tacoma-Pierce  County 

Department  of  Health,  and  Department  of  the  Army  -  Madigan 
Army  Medical  Center.  All  letters  indicated  support  for  a  broad- 
based  community  approach  to  assist  in  determining  requirements 
for  data  collection  and  data  sharing  across  respective 
jurisdictions. 

Task  1  a.  Create  an  executive  management  team  of  key  Completed:  Initial  management  team  created  in  Fall  2003; 

stakeholders  including  investigators,  key  participants  and  Reorganized  management  team  became  effective  August 

administrative  resources. (Month  1)  2006:  The  initial  management  team  was  identified  during  the  first 

month  of  the  Project.  (See  Appendix  A  for  original  names,  of 
titles  and  organizational  affiliations.)  As  was  noted  in  previous 
reports,  during  the  course  of  these  meetings,  several  technical  and 
policy  issues  were  identified.  In  order  to  better  address  the  issues, 
a  chartering  process  was  undertaken.  At  the  conclusion  of  the 
chartering  process  in  December  2006,  the  management  team 
identified  is  as  follows:  Washington  State  Department  of 
Health:  Jude  Van  Buren,  Dr.,  PH,  Assistant  Secretary  of  Health; 
Kitsap  County  Department  of  Health:  Scott  Lindquist,  MD, 
Health  Officer;  Tacoma-Pierce  County  Health  Department: 
Vic  Harris,  PhD,  Deputy  Director;  Foundation  for  Health  Care 
_ Quality:  Dr.  Peter  Dunbar,  PI,  Andy  Fallat,  CEO,  FHCQ;  and 
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Task  1  b.  Identify  potential  participant  sites  to  include  the 
principal  emergency  departments  in  the  participating  counties. 
These  sites  will  be  representative  of  the  “Sea-Tac”  corridor  and 
will  include  facilities  contiguous  to  the  Ft.  Lewis,  McChord 
A.F.B,  Bremerton  Naval  Yard,  Everett  Home  Port,  and  the  NAS 
Whidbey  Island.  (  Month  1) 


Paladin  Data  Systems:  Gary  Macy,  Chief  Technical  Officer  (See 

Appendix  B  for  complete  copy  of  ODIN  Health  Data 
Surveillance  and  Analysis  Toolkit  Project  Charter) 


Completed  August  2006:  Preliminary  sites  were  identified  in 
Fall  2003;  at  the  conclusion  of  the  chartering  process, 
participating  organizations/  sites  include:  Washington 
Department  of  Health,  Tacoma-Pierce  County  Health 
Department,  and  Kitsap  County  Health  District.  These 
jurisdictions  include  principle  emergency  departments  sites  in 
participating  counties  and  include  military  facilities  at  Fort  Lewis, 
Bremerton  Naval  Base. 


Task  1  c.  Implement  a  joint  application  development  process  Completed:  A  joint  application  development  process  was 
using  identified  subject  matter  experts  to  identify  and  document  implemented  within  the  initial  3  months  of  the  project.  There 
data  and  system  requirements  for  the  web-based  data  collection.  were  unanticipated  difficulties  in  directly  obtaining  the  time 

required  from  the  local  public  health  officials  to  identify 
requirements.  In  addition  different  local  health  jurisdictions  used 
different  methodologies  for  syndromic  surveillance,  and  obtaining 
agreement  of  requirements  would  have  been  challenging. 

We  found  that  we  could  successfully  obtain  good  requirements 
without  direct  access  to  end  users  for  requirement  elicitation  by 
the  development  team  by  using  a  super-user  approach  (///).  The 
super-user  approach  involves  using  a  surrogate  user  in  place  of 
actual  end  users  when  end  users  are  unavailable  for  direct 
consultation.  For  this  approach  to  work  the  surrogate  user  must 
have  domain  knowledge  and  have  knowledge  of  how  end  users 
operate. 

The  super-user  obtained  the  knowledge  needed  to  generate  good 
requirements  through 

1 .  Interactions  with  the  local  health  jurisdictions 
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participating  in  the  project, 

2.  Interactions  with  project  participants  at  the  University  of 
Washington  School  of  Public  Health. 

3.  Conducting  a  review  of  and  developing  expertise  in 
current  syndromic  surveillance  systems  (including  CDC’s 
BioSense  project,  Electronic  Surveillance  System  for  the 
Early  Notification  of  Community-Based  Epidemics 
(ESSENCE),  Realtime  Outbreak  and  Disease  Surveillance 
(RODS),  Over-the-Counter  drugs  and  other  items  (OTC), 
and  Early  Aberration  Reporting  System  (EARS),  using 
access  provided  by  the  local  health  jurisdictions, 

4.  Systematic  review  of  the  syndromic  surveillance 
literature, 

5.  Attending  and  presenting  at  academic  and  industry 
conferences 

The  super  user  interacted  with  subject  matter  experts  by  making 
himself  available  as  a  PhD  Biostatistician  that  the  local  health 
jurisdictions  could  use  as  a  resource.  This  enabled  the  super-user 
to  observe  what  challenges  were  facing  the  end  users. 

We  found  that  the  joint  application  process  worked  well  in  terms 
of  anticipating  technical  requirements  and  operational  patterns. 
The  need  for  involvement  with  the  stakeholder  organizations  by 
the  “super  user’1  lessoned  as  more  the  requirements  of  the  system 
are  understood.  However,  it  should  be  noted,  without  direct 
involvement  of  the  stakeholder  organizations  it  was  difficult  to 
prioritize  development. 

Appendix  C  lists  the  requirements  developed  from  the  charter 
process  together  with  the  status  of  these  requirements  in  the 
development  process  of  ODIN  at  the  start  of  the  charter  process. 
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The  data  access  model  for  ODIN  was  designed  to  meet  security 
and  privacy  issues.  No  “limited  data  set”  data  is  stored  within 
Identify  all  necessary  mechanisms  to  ensure  the  security  and  ODIN.  Access  to  encounter  level  details  is  subject  to 
privacy  of  proposed  data  collection  is  consistent  with  HIPAA  authorization  by  the  jurisdiction  that  the  data  was  collected  in. 
requirements.  (Months  1-3)  Access  to  aggregate  data  can  be  given  to  a  user  while  withholding 

access  to  encounter  level  data. 


Task  1  d.  Draft  detailed  requirement  documentation  including  We  found  that  these  standards  did  not  impact  core  system 
logical  data  models,  use  cases,  activity  diagrams  and  functional  development.  Outside  interfaces  within  ODIN  were  designed  to 
specifications,  and  review  with  all  participants,  stakeholders  and  be  compatible  with  other  systems, 
potential  users.  This  would  include  the  reconciliation  of 

requirements  with  pertinent  data,  privacy  and  other  standards,  The  ODIN  system  was  designed  to  be  hosted  by  public  health 
including  National  Health  Information  Infrastructure  (NHII)  and  organizations  that  have  in  place  the  necessary  privacy  and 
National  Electronic  Disease  Surveillance  System  (NEDSS).  security  framework  for  access  to  sensitive  data.  The  system  is 

designed  to  reside  behind  the  host  organizations  firewall. 

Logical  data  models  were  crafted  by  the  IT  contractor. 
Requirements  documents  were  generated  and  consisted  of 
functional  specifications  (including  screen  shots)  and  use  case 
(activity)  diagrams. 

Prototypes  of  the  requirements  were  implemented  in  monthly 
sprints,  and  requirements  modified  based  on  test  usage  of  the 
designs  and  technical  considerations.  Due  to  the  restrictions  on 
direct  access  to  potential  users  (see  lc),  in  progress  requirements 
were  not  shared  with  potential  users.  Instead  demonstrations  of 
the  phase  I  prototype  were  used  to  gain  user  feedback.  Feedback 
from  these  demonstrations  showed  that  the  requirements 
generated  through  the  super-user  approach  matched  the  needs  of 
the  local  health  jurisdictions. 
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Create  prototype  models  to  test  the  functional  acceptances  and 
review  the  prototype  models  or  the  proposed  system  with  all 
participants,  then  re-craft  requirements  based  on  prototype 
review.  (Months  3-5) 

Two  prototypes  for  ODIN  was  created  (called  the  phase  1 
prototype  and  the  phase  II  prototype.).  We  found  that  both 
prototypes  had  the  technical  capacity  to  host  at  least  2  years  worth 
of  encounter  data  for  the  state  of  Washington,  roughly  1-2  million 
patient  encounters.  Two  years  worth  of  encounter  data  for  the 

State  of  Washington  was  simulated  and  loaded  into  the  prototypes 
with  no  impact  on  performance. 

Demonstrations  were  given  to  each  local  health  jurisdiction,  the 
Washington  State  Department  of  Health  and  the  army,  and 
feedback  in  terms  of  changes  and  additional  requirements 
collected.  We  found  that  the  prototypes  contained  sufficient 
functionality  to  meet  the  basic  needs  of  the  health  jurisdictions. 

The  prototype  was  also  shown  at  the  (Puerto  Rico  Conference), 
but  could  not  be  demonstrated  iive'  due  to  technical  limitations 
at  the  conference  site.  /2/  See  appendix  D 

Task  2  -  Case  Reporting 

Status-  Research  Accomplishments 

Task  2  -  Case  Reporting  -  Rapidly  develop  a  web-based  case 
reporting  system  for  syndromic  clinical  data,  and  implement  that 
system  in  a  variety  of  healthcare  organizations  throughout  a  four 
County  area  in  Western  Washington  region.  (Months  5-24) 

See  Appendix  E  dated  12/23/03  documenting  that  the 
Statement  of  Work  remains  an  accurate  description  of  the 
proposal.  The  four  Sub-tasks  under  Task  2  remain  relevant 
except  that  they  will  be  applied  to  other  collection  methods 
consistent  with  ESSENCE. 

Public  Health  leaders,  both  local  and  national,  were  found  to  be 
highly  skeptical  of  a  web-based  system  approach.  AIBS  Peer 
Review  to  USAMRMC  identified  this  component  of  the  study  as 
a  weakness.  Public  health  leadership  recommended  a  more 
reliable  and  affordable  approach  to  developing  an  accurate 
information  system  for  syndromic  surveillance.  Public  health 
officials  requested  the  project  focus  on  developing  techniques 
with  reasonable  expectations  for  the  long-term  value.  As  reported 
in  March  2005,  based  on  immediate  feedback  on  this  approach, 
the  web-based  case  reporting  system  is  not  being  pursued.  This 
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decision  does  not  require  a  change  in  the  Statement  of  Work. 

We  found  that  a  web  based  case  reporting  system  was  strongly 
indicated  against  by  the  project  participants.  During  the  run-up  to 
the  start  of  the  grant  a  shift  away  from  case  reporting  systems 
occurred.  While  there  are  documented  positive  benefits  to  a  case 
reporting  system  (develop  relationships  with  hospital  staff,  easy 
to  initiate,  detailed  information  obtainable)  and  it  has  been 
successfully  used  on  a  drop  in  basis,  /i/  However,  it  is  widely 
recognized  as  labor  intensive  and  difficult  to  maintain,  and 
experience  has  shown  it  to  be  not  a  sustainable  form  of 
surveillance  / 4j 

Task  2  a.  Develop  and  review  technical  specifications  to  support 
detailed  requirements  identified  by  participants,  stakeholders  and 
users.  Develop  plan  for  unit  and  acceptability  testing.  Develop 
use  cases  from  functional  specifications  to  guide  development 
and  testing  (Months  5-7) 

See  lc. 

Develop  use  cases  from  functional  specifications  to  guide 
development  and  testing  (Months  5-7) 

We  found  that  the  rapid  prototyping  approach  worked  well  in  the 
development  environment.  Logical  data  models  were  crafted  , 
requirements  documents  were  generated  and  consisted  of 
functional  specifications  (including  screen  shots)  and  use  case 
(activity)  diagrams. 

Prototypes  of  the  requirements  were  implemented  in  monthly 
sprints,  and  requirements  modified  based  on  test  usage  of  the 
designs  and  technical  considerations.  Due  to  the  restrictions  on 
direct  access  to  potential  users,  in  progress  requirements  were  not 
shared  with  potential  users.  Instead  demonstrations  of  the  phase  I 
and  phase  II  prototypes  were  used  to  gain  user  feedback. 
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Feedback  from  these  demonstrations  showed  that  the 
requirements  generated  through  the  super-user  approach  matched 
the  needs  of  the  local  health  jurisdictions. 

Testing  system  using  simulated  data:  A  large  data  set  was 
simulated  and  used  to  test  the  functionality  and  capacity  of  the 
system.  The  data  consisted  of  simulated  data  over  a  2  year  period 
for  the  State  of  Washington  (a  total  of  834564  encounters).  The 
simulation  was  designed  to  provide  data  with  similar 
characteristics  to  actual  syndromic  surveillance  data,  and  included 
nuances  of  missing  data,  subtle  and  gross  errors  in  data,  simulated 
outbreaks  and  other  data  anomalies.  The  simulated  data 
demonstrated  the  capacity  of  ODIN  system  to  handle  data  of  at 
least  twice  the  anticipated  final  data  size,  and  successfully 
demonstrated  the  implementation  of  the  functionality.  In  addition, 
it  provided  a  convenient  test  bed  for  iterative  requirements 
development. 

Findings  from  the  requirements  directed  the  technical 
specifications.  Key  findings  from  the  requirements  process 
include: 

1 .  Algorithms  should  not  be  fixed,  as  different  preferences  and 
evolving  development  of  algorithms,  flexibility  of  algorithm 
implementation  is  key. 

2.  Geographic  units  should  be  flexible  and  multilevel.  For 
example,  a  large  proportion  of  Kitsap  County  is  covered  by  a 
single  zip  code,  so  that  zip  codes  do  not  provide  sufficient 
geographic  detail  for  spatial  analyses  in  this  county.  However 
King  County  is  covered  by  over  100  zip  codes,  and  zip  codes 
provide  too  much  detail  for  many  purposes. 
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Task  2  b.  Perform  assessments  of  workflow  and  technical 
capacity  at  participating  sites  to  select  an  option,  and  then 
document  implementation  plan,  both  to  assure  implementation  of 
success  as  well  as  to  characterize  variations  in  operational 
patterns,  personal  technical  capacity  and  other  key 
implementation  issues.  (Months  7-9) 


3.  A  multi-tiered  security  model  is  needed  for  implementation  of 
a  system  across  multiple  jurisdictions. 

4.  Multiple  usage  pathways  are  required  to  support  the  different 
business  processes  that  the  local  health  jurisdictions  have. 

5.  The  interpretation  of  data  and  alerts  depends  on  local 
knowledge  about  the  nature  of  the  data. 


Assessment  of  workflow  and  technical  capacity  at  the 
participating  sites  was  completed.  Based  on  assessment  and  input 
from  sites,  the  system  is  being  designed  to  support  a  number  of 
tools  as  well  as  allow  the  user  to  develop  their  own  tools  and 
extensions.  The  system  has  the  functions  listed  below: 

1 .  Securely  encrypt,  transmit,  stage,  process,  store  and 
present  data  using  data  warehousing  industry’s  best 
practices 

2.  Plug-in  capability  for  interchangeable  tools  or  methods  to 
group  (unstructured  text  chief  complaint  data  into 
syndromic  classes 

3.  Plug-in  capability  for  interchangeable  tools  or  methods  to 
analyze  the  data  and  detect  anomalies  and  patterns 

4.  Present  data  analysis  result  summaries  using  table,  graph  and 
geo-spatial  map  visualizations  and  drill  down  capability. 

5.  Automatically  trigger  anomaly  alerts  based  on  detection 
findings  and  queue  notification  messages  for  distribution. 

A  key  finding  is  that  each  of  the  local  health  jurisdictions  has 
different  capabilities  for  surveillance  and  different  work  flow 
processes. 
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We  found  that  the  super-user  approach  allowed  us  to  develop  an 
adaptable  system  to  meet  the  requirements  of  the  different  work 
flow  processes.  We  found  during  demonstrations  of  the  phase  I 
and  phase  II  prototypes  to  potential  end  users  that  almost  all 
current  and  future  needs  had  been  incorporated,  or  were  planned 
to  be  incorporated,  into  the  ODIN  system. 

We  found  that  there  was  a  need  for  more  in  depth  research  into 
the  work  flow  processes  within  local  health  jurisdictions,  and 
arranged  for  this  work  to  be  carried  out  by  the  University  of 
Washington  through  the  subcontract.  The  results  of  this  research 
are  pending . 

Task  2  c.  Implement  the  case  reporting  application,  and  perform 
unit  and  acceptance  testing  on  both  the  web  application  and  the 
central  server.  (Months  8-11) 

Answered  in  lc. 

Task  2  d.  Implement  the  system  at  participating  sites,  and 
provide  ongoing  follow-up  assessment,  support  and 
documentation  of  implementation  issues  through  the  use  of  third 
party  web-deployed  issues  tracking  tools,  and  through  contract 
reporting  (Months  1 1-24) 

Completed  December  2006:  As  reported  earlier,  implementation 
of  system  prototype  testing  at  participating  sites  was  originally 
planned  for  June/July  2005.  Due  to  issues  raised  regarding  control 
of  data,  control  of  the  project  and  related  liability  issues  of  for- 
profit  company  as  a  contactor  for  software  development  and 
concerns  regarding  access  to  protected  health  information,  there 
was  a  delay  in  testing  the  at  participating  sites.  As  part  of  the 
decision  making  associated  with  the  Chartering  process, 
participates  agreed  to  test  the  system  at  their  respective  sites.  As 
of  March  2007  the  following  sites  are  participating  in  the  pilot 
implementation.  Washington  State  Department  of  Health, 
Kitsap  County  Department  of  Health,  and  Tacoma-Pierce 
County  Health  Department. 
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Task  3  -  Automated  Data  Collection/Integration 


Status-  Research  Accomplishments 


Task  3  -  Automated  Data  Collection/Integration  -  Data  will  be 
collected  from  the  sentinel  healthcare  organizations  in  all 
participating  counties,  with  the  direct  reporting  of  data  to  the  local 
health  jurisdictions.  Integrate  the  data  from  the  web-based  case 
reporting  system,  the  automated  data  collection  system,  and  the 
military’s  public  health  surveillance  system,  the  support 
appropriate  access  by  both  military  and  civilian  health  authorities. 
(Months  1-24) 


As  noted  in  the  March  2005  report,  the  system  design 
specifications  and  scope  documents  identify  the  approach  and 
methodology  to  be  used  for  Automated  Data  Collection  and 
Integration.  The  system  was  designed  to  incorporate  a  "smart" 
rules  based  data  collection  engine  which  is  capable  if  collecting 
data  in  a  variety  of  formats  from  a  wide  range  of  organizations. 

We  found  that  it  was  not  practical  to  collect  'live’  data  while 
system  development  was  in  progress.  In  particular  we  found  that 
due  to  concerns  by  public  health  jurisdictions  about  jeopardizing 
their  relationships  with  local  hospitals,  they  were  not  willing  to  let 
live  data  be  put  into  a  system  under  development  and  hosted 
outside  of  the  jurisdictions. 


We  found  that  we  could  use  simulated  data  developed  according 
to  specifications  provided  by  the  local  health  jurisdictions  and 
based  on  published  work  for  testing  and  development  purposes 
for  all  parts  of  the  system  except  for  the  data  import  processes. 

Task  3  a.  In  concert  with  the  Community  Initiative,  develop  Completed  December  2006:  The  Charter  is  the  formal  reference 
technical  agreements  with  the  IT  groups  of  sentinel  sites  for  shared  assumptions  intended  to  guide  and  define  the  ODIN 
healthcare  organizations  with  all  counties,  to  cover  data  elements,  project.  As  noted  in  the  Charter,  State  of  Washington  Department 
coding,  security,  and  service-level  agreements.  Develop  similar  of  Health  agreed  to  “host”  the  ODIN  technical  system,  perform 
agreements  with  Madigan  for  centralized  exchange  of  ESSENCE  project  management  activities  and  coordinate  all  activities  with 
II  or  similar  regional  military  data.  (Months  1-12)  the  two  local  health  jurisdictions  Kitsap  County  Health  District 

and  Tacoma-Pierce  County  Health  Department  including 
completing  agreements  to  cover  data  elements,  coding,  security 
and  service  level  agreements  as  required.  The  overall  approach  is 
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Task  3  b.  Extend  our  present  server  architecture  to  accommodate 
required  scaling,  add  additional  site-specific  normalizations,  and 
extend  query  structure  to  include  multi-jurisdictional  data  access. 
(Months  1-6) 


to  compare  two  syndromic  surveillance  software  applications  in 
their  ability  to  gather,  process  and  provide  disease  incidence  data 
on  which  to  base  public  health  response  decisions.  The 
emergency  room  facilities  within  Kitsap  County  and  Tacoma- 
Pierce  County  currently  report  disease  data  in  a  automated 
surveillance  application  (ESSENCE),  that  data  will  be  copied  and 
sent  through  the  ODIN  process  algorithms  and  made  available  at 
the  password  protected  web-based  portal  for  use  by  the  county 
public  health  systems.  The  data  from  the  two  systems  will  be  used 
to  inform  the  public  health  staff.  Additional  funding  ($3.6  M) 
was  requested  to  continue  the  technical  development,  develop  and 
validate  additional  statistical  techniques,  train  staff,  and  continue 
to  extend  the  technical  system  capabilities. 

An  evaluation  will  be  conducted  through  December  2007  to 
provide  feedback  on  the  advantages  and  challenges  of  both 
systems.  (See  Appendix  F  for  Agreement  between  Foundation 
for  Health  Care  Quality  and  State  of  Washington  Department 
of  Health) 


The  system  design  specification  is  scalable  both  in  capacity  and 
capability.  Based  on  industry  standard  data  management 
products,  the  system  uses  self-defining  data  structures  throughout 
its  design,  allowing  for  future  expansion  and  addition  of  data 
sources  without  requiring  a  programmatic  change.  Access  to  the 
information  in  the  system  is  controlled  by  a  robust  security 
protocol  which  provides  granular  access  management  and  system 
journaling. 


We  found  that  there  were  no  technical  difficulties  in 
implementing  the  scalability;  the  system  was  designed  from  the 
bottom  up  to  be  scalable  and  cover  multiple  jurisdictional  needs. 
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Task  3  c.  Implement  automated  data  collection,  easing  the 
present  3-tier,  HIPAA  compliant  data  model  at  the  sentinel  sites. 
We  anticipate  being  able  to  implement  at  multiple  site  in  each 
participating  county  over  this  period.  (Months  3-21) 


(Analysis  and  recommendation  of  HIPAA  application  provided  by 
John  R.  Christiansen,  Christiansen  IT  Law,  a  law  firm 
specializing  in  technology  related  services) 

Scope  of  data  available  for  contribution  to  ODIN:  HIPAA 
prohibits  disclosure  of  Protected  Health  Information  for  most 
public  health  purposes,  unless  individual  authorization  is 
obtained.  Obtaining  such  authorization  may  be  difficult  or 
sometimes  impossible.  However,  HIPAA  expressly  does  permit 
disclosure  of  “Limited  Data  Sets”  for  public  health  purposes 
without  such  authorization.  Limited  Data  Sets  appear  to  provide 
sufficient  information  for  ODIN  purposes.  Authority  to  use  data 
processing  vendors  to  provide  ODIN  data:  Many  hospitals  rely 
upon  data  processing  services  vendors  to  store  and  manage  their 
Protected  Health  Information.  However,  HIPAA  requires  Covered 
Entities  (including  hospitals)  to  limit  the  uses  and  disclosures 
such  vendors  may  make  of  their  Protected  Health  Information. 
Model  contract  provisions  authorizing  vendors  to  create  and 
disclose  Limited  Data  Sets  on  behalf  of  hospitals  for  public  health 
purposes,  including  ODIN,  have  been  developed  and  found 
acceptable  to  both  a  major  data  processing  services  vendor  and  a 
number  of  hospitals.  (See  Attachment  D) 


The  system  has  been  implemented  at  the  participating  local  health 
jurisdictions  and  health  care  organizations. 

We  found  that  considerable  organizational  challenges  around  data 
sharing  along  with  competing  efforts  prevented  implementation 
of  the  ODIN  system  at  sites  until  these  issues  are  resolved. 


Currently  all  planned  surveillance  capabilities  within  ODIN  make 
use  of  limited  data  sets.  There  are  no  direct  legal  impediments  to 
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local  public  health  jurisdictions  receiving  limited  data  set  data 
from  hospitals.  [5/ 

There  is  however  the  potential  for  hospitals  to  be  cautious  about 
sharing  limited  data  set  information  with  public  health 
jurisdictions  due  to  perceptions  of  HIPAA  related  concerns  [6] 


Task  3  d.  Develop  and  implement  support  for  bi-directional 
exchange  of  data  with  military  systems,  working  at  either  the 
regional  level  with  Information  Technology  staff  at  Madigan 
Army  Medical  Center,  or  at  the  national  level  with  the  ESSENCE 
staff  (Months  6-12) 


There  has  been  an  ongoing  communication  with  key  points  of 
contacts  with  military  systems  since  the  beginning  of  the  ODIN 
project. 

Julie  Pavlin,  M.D.,  M.P.H.,  Walter  Reed  Army  Institute  of 
Research  initially  served  on  the  Steering  Committee.  After  the 
ESSENCE  project  was  transferred  to  Health  Affairs,  Col  Kenneth 
L  Cox,  Director,  Force  Health  Readiness,  Health  Affairs  replaced 
Dr.  Pavlin. 

Dr.  Peter  Dunbar,  PI  and  Linda  Lekness  participated  in  a 
conference  call  with  Col.  Cox  on  2/3/05.  Col.  Cox  expressed 
interest  and  support  for  the  project.  Additional  communication 
with  Col  Kenneth  Cox  occurred  on  4/13/05;  5/05;  7/31/05  and  on 
8/25/05  Dr.  Dave  Ross,  Consultant  to  the  ODIN  project, 
completed  a  phone  interview  with  Col  Cox  as  part  of  Dr.  Ross’s 
effort  to  complete  a  needs  assessment  of  users  of  the  ODIN 
system.  On  1/25/06  Dr.  Bud  Nicola  and  Laura  Ripp  completed  a 
phone  interview  with  Col  Cox  as  part  of  the  charter  development 
process.  As  “lessons  learned”  become  available  from  other 
similar  projects,  Col  Cox  has  offered  to  provide  that  information 
to  the  ODIN  project. 


Additional  coordination  with  military  systems  include:  Dr.  Peter 
Dunbar  met  with  Lieutenant  Colonel  Andrew  Wiesen,  MD,  PhD, 
Clinical  Assistant  Professor,  Madigan  Army  Medical  Center, 
Department  of  Preventive  Medicine  in  November  2004.  LTC 
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Wiesen  stated  that  since  Madigan  was  already  reporting  data  to 
ESSENCE  and  that  once  the  project  had  reached  the  appropriate 
milestones  he  would  be  interested  in  working  out  additional 
details.  On  May  13,  2005  Lt.  Col  Andrew  Wiesen  participated  in 
a  presentation  of  the  ESSENCE  system  to  ODIN  stakeholders 
including  Dr.  Jude  Van  Buren,  Assistant  Secretary,  Washington 
State  Department  of  Health,  Dr.  Ian  Painter,  Biostatistician  and 
Linda  Lekness.  Lt  Col  Wiesen  was  provided  a  brief  status  report 
at  this  time. 

As  noted  in  Task  la  status,  written  monthly  progress  reports  have 
been  submitted  to  the  military  points  of  contacts  for  the  ODIN 
project. 

Finally  the  project  team  has  continued  to  stay  in  touch  with 
professionals  across  the  country  working  on  similar  projects.  For 
example,  BioDefend  ,  a  project  between  the  University  of  South 
Florida’s  Center  for  Biological  Defense  and  Datashpere,  LLC 
have  developed  a  system  for  conducting  bioterrorism  and 
infectious  disease  surveillance.  The  BioNet  project  was  also 
contacted  but  working  with  this  project  was  not  feasible  due  to 
the  work  being  classified.  This  project  is  also  working  toward  bi¬ 
directional  exchange  of  data  with  military  systems,  working  at 
either  the  regional  level  or  national  level  in  coordination  with  Col 
Ken  Cox.  As  of  March  2007,  bi-directional  exchange  of  data  has 
not  been  implemented.  This  effort  will  continue  to  be  pursued 
with  additional  funding  received. 


Task  3  e.  Continue  to  improve  and  maintain  centralized  We  found  that  establishing  the  system  in  the  Department  of 
integration  server,  and  continue  to  improve  and  maintain  secure  Health  and  the  commencement  of  data  flow  into  the  system  has 
data  transmission  using  both  accepted  and  evolving  standards  for  renewed  interest  by  local  public  health  agencies  in  investigating 
security  and  message  protocols.  (Months  12-24.)  the  inclusion  of  alternative  data  sources  for  surveillance  purposes. 
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Task  4  -  Utility  Assessment/Detection  and  Visualization 


Status-  Research  Accomplishments 


Task  4  -  Assess  the  individual  and  combined  utility  of  web-based  AIBS  Peer  Review  to  USAMRMC  identified  the  component  of 
case  reporting  and  automated  data  collection  on  the  same  assessing  web  based  reporting  as  a  weakness  in  the  project.  Based 
populations  through  application  of  existing  algorithms  and  on  immediate  feedback  on  this  approach,  the  web-based  case 

visualizations.  (Month  1-24)  reporting  system  is  not  being  pursued.  See  Appendix _ dated 

12/23/03  documenting  that  the  Statement  of  Work  remains  an 
accurate  description  of  the  proposal.  The  sub-tasks  under  Task  4 
remain  relevant  as  reported  below. 

Task  4  a.  Gather  algorithms  inclusion  criteria:  use  in  a  Through  a  comprehensive  review  of  the  issues  in  algorithm 
bioterrorism  or  epidemic  detection  system.  development  and  implementation  we  found  that  the  algorithm 

development  field  for  syndromic  surveillance  had  not  fully 
matured,  and  that  no  consensus  on  best  practice  algorithms 
existed,  and  that  considerable  development  effort  was  going  into 
producing  better  algorithms.  Accordingly  we  design  the  ODIN 
system  to  allow  multiple  algorithms  to  be  included. 

Task  4  b)  Test/evaluate  the  performance  of  the  above  approaches  We  also  found  that  there  are  considerable  uncertainties  about 
against  set  historical  outbreaks  and  novel  outbreaks.  whether  the  performance  of  an  algorithm  (or  syndrome  coder)  on 

one  set  of  data  will  reflect  the  actual  performance  on  a  different 
set  of  data,  that  is  to  say,  there  may  be  considerable  local 
variability  in  performance.  The  University  of  Washington  through 
its  subcontract  developed  a  system  using  a  web  based  services 
model  that  enables  analysis  of  algorithm  performance  to  be 
conducted  over  the  web  against  standard  datasets  and  outbreak 
models  without  the  technical  implementation  knowledge  typically 
needed  to  conduct  these  simulations.  Given  the  flexibility 
requirements  of  algorithm  implementation  within  a  multi- 
jurisdictional  syndromic  surveillance  system,  and  the  requirement 
that  local  knowledge  of  data  patterns  is  required  that  we  found 
(see  section  2a),  a  web  based  services  model  that  allows  the 
analysis  of  algorithm  performance  against  local  data  is  important. 
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One  of  us  (Dr  Painter)  also  was  involved  in  a  large  simulation 
study  to  systematically  look  at  the  performance  characteristics  of 
several  commonly  use  algorithms  within  a  large  public  health 
jurisdiction  /7/. 

Task  5  -  Ethnographic  Analysis 

Status-  Research  Accomplishments 

Task  5  -  Ethnographic  Analysis  -  Appraise  the  value  of 
syndromic  clinical  data  to  decision  makers  in  public  health  and 
disaster  management  by  conducting  structured,  retrospective 
assessments  of  the  personnel  involved  with  three  recent  events: 
the  ongoing  smallpox  vaccination  program,  the  SARS  outbreak, 
and  the  2003  Seattle  TOPOFF  exercise.  (Months  1-24) 

As  part  of  the  condition  of  participation  in  this  project  by  Public 
Health,  ethnographic  analysis  was  not  pursed. 

Task  5  a.  Dissemination  of  Results  -  All  investigators  will  seek  to 
present  intermediate  and  final  results  through  conference 
presentation  and  peer  review  publication.  This  is  anticipated  to 
begin  in  month  6,  based  on  extension  of  prior  work,  and  extend 
past  the  end  of  the  contact.  (Months  6-36) 

Results  dissemination  through  posters  and  presentations.  See 
reportable  outcomes. 

Task  5  b.  Project  Reporting  -  All  investigators  will  assist  the 
Principal  Investigator  in  providing  reporting  as  required  to  satisfy 
contact  terms  (Months  1-24) 
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Key  Research  Accomplishments: 

•  We  developed  a  formal  reference  document  (charter)  that  enabled  cross-jurisdictional  cooperation  in  organizing  a  multi- 

jurisdictional  surveillance  system  (task  1): 

o  We  found  that  there  are  considerable  organizational  and  political  barriers  to  developing  and  employing  cross-jurisdictional 
surveillance  systems. 

o  We  found  that  these  organizational  and  political  barriers  could  be  successfully  navigated  a  formal  chartering  process, 
o  We  developed  a  final  signed  charter  document  and  on  the  basis  of  this  document  installed  a  state  of  the  art  surveillance 
system  at  the  Washington  State  Department  of  Health. 
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•  We  designed,  developed  and  installed  a  state  of  the  art  syndromic  surveillance  system  that  accepts  multiple  data  sources  and 
provides  a  region-wide  view  of  information  in  the  system  (tasks  2,  3  and  4): 

o  We  found  that  a  rapid  application  development  approach  together  with  a  super-user  could  successfully  develop  such  a 
system  without  direct  access  to  end  users. 

o  We  found  that  there  were  no  technical  difficulties  in  developing  the  system,  in  part  due  to  the  implementation  of  a  solid 
and  flexible  system  and  database  architecture. 

o  We  found  that  advanced  visualization  tools  could  be  delivered  over  the  web  using  a  web  browser  as  client. 

o  We  developed  several  advanced  visualization  tools  designed  to  overcome  screen  real-estate  limitations  when  displaying 
multi-level  data. 

o  We  found  that  in  general  the  development  of  appropriate  analytical  tools  is  in  an  immature  state,  and  novel  tools  with 
improved  performance  are  expected  to  emerge.  For  this  reason  we  focused  on  building  in  flexibility  in  the  selection  and 
implementation  of  analytical  tools  into  the  system. 

•  ODIN  stakeholder  collaboration  with  the  University  of  Washington  School  of  Public  Health  and  Community  Medicine  actively 
supported  the  development  of  the  Center  for  Excellence  in  Public  Health  Informatics  at  the  University  of  Washington  (tasks  2b) 

o  The  Centers  for  Disease  Control  and  Prevention  awarded  a  grant  in  the  amount  of  $3.8  million  to  the  University  of 

Washington  School  of  Public  Health  and  Community  Medicine  to  support  the  development  of  a  Center  for  Excellence  in 
Public  Health. 

o  This  center  will  continue  to  advance  the  science  of  syndromic  surveillance  and  public  health  informatics  in  general,  long 
after  the  ODIN  project  is  completed. 

o  The  center  will  continue  to  build  on  the  collaboration  between  and  among  the  Department  of  Defense  and  the  civilian 
Public  Health  community. 


Reportable  Outcomes:  (and  associated  tasks) 

Posters  and  Presentations: 

•  Bliss  D,  Karras  BT.  "Interactive  3D  Visualization  of  Public  Health  Geographic  Data  Using  Google  Earth"  [computer 
demonstration]  4th  Public  Health  Information  Network  (PHIN)  Stakeholders  Conference,  Atlanta,  GA.  2006  Sept  (task  4) 
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•  Boyce  RD,  Karras  BT,  Lober  WB.  Pre-processing  to  improve  the  classification  of  chief  complaint  data,  [paper/oral 
presentation]  Syndromic  Surveillance  Conference,  Seattle,  WA.  2005  Sept  (task  4) 

•  Doctor  J,  Baseman  J,  Karras  BT,  Lober  WB.  “Multi-attribute  Utility  Theory  for  Prioritizing  Data  Elements  for  Surveillance 
using  Regional  Health  Information  Organization  Data"  [poster]  ( task  3a) 

•  Dunbar,  PJ,  IS  Painter,  L  Lekness  (2006)  Puget  Sound  Infectious  Disease  Tracking  System.  Poster  and  demonstration 
presented  at  the  Department  of  Defense  Military  Health  Research  Forum,  Puerto  Rico,  May  2006.  (all  tasks) 

•  Karras  BT,  “Detecting  emerging  diseases”  [Invited  Plenary  Speaker],  Sichuan  University,  Chengdu,  China  2006  Sept  (all 
tasks) 

•  Karras  BT,  Detmer  D,  The  Convergence  of  Public  Health  and  Biomedical/Health  Informatics,  4th  Public  Health  Information 
Network  (PHIN)  Stakeholders  Conference,  Atlanta,  GA.  2006  Sep  (task  3) 

•  Karras  BT,  “Development  and  Utilization  of  Competencies  for  Public  Health  Informatics”  [Panel  organizer  and  presenter]  4th 
Public  Health  Information  Network  (PHIN)  Stakeholders  Conference,  Atlanta,  GA.  2006  Sept  (all  tasks) 

•  Lober  WB,  Mandl  K,  Mostashari  F,  Overhage  JM,  Wagner  M,  Modern  Health  Surveillance,  [Panel  Organizer/Presenter] 

AMI  A  Symposium,  Washington  DC,  Nov  2005.  (all  tasks) 

•  Lober  WB,  Oberle  M,  Mandl  K,  Platt  R,  Making  Regional  and  National  Health  Information  Exchanges  Work  for  Public 
Health:  The  Vision  of  the  CDC  Centers  of  Excellence  in  Public  Health  Informatics,  [Panel  Presentation]  4th  Public  Health 
Information  Network  (PHIN)  Stakeholders  Conference.  Atlanta,  GA.  2006  Sept,  (all  tasks) 

•  Painter  I,  A  Harvey,  K.  Peterson,  K.  Welling,  K.  Taylor,  P.T.  Tran  (2006)  Some  Interactive  Data  Exploration  Tools  for 
Syndromic  Surveillance.  Poster  presented  at  the  Syndromic  Surveillance  conference  Baltimore  2006.  (task  4) 

•  Painter  I,  K  Sebestyen,  T  Lumley  (2005)  Effects  of  coder  misclassification  on  outbreak  detection.  Poster  presented  at  the 
Syndromic  Surveillance  conference  Seattle  2005.  (task  4b) 

•  Wagner  S,  Karras  BT.  Quality  Analysis  of  Syndromic  Surveillance  Data  [poster  presentation]  4th  Public  Health  Information 
Network  (PHIN)  Stakeholders  Conference.  Atlanta,  GA.  2006  Sept  (task  4) 


Page  23  of  91 


Peer  reviewed  publications: 


•  Karras  BT,  Bliss  D,  Lober  WB,  Horn  S,  Lindquist  S,  Information  Collection  SSIC-GeoCodes  for  urban  to  rural  mixed 
environments,  Advances  in  Disease  Surveillance  2006;  1:38 

•  Jackson,  M.L.,  A.  Baer,  I.  Painter  and  J.  Duchin  (2007)  A  simulation  study  comparing  aberration  detection  algorithms  for 
syndromic  surveillance.  BMC  Medical  Informatics  and  Decision  Making  2007,  7:6  ([7]) 

•  Lober  WB,  Drozd  D,  Lumley  T,  Sebestyen  K,  An  Open  Source  Web  Services  Toolkit  for  Event  Detection  Algorithms, 
Advances  in  Disease  Surveillance  2006;  1:78 

•  Lumley  T,  Sebestyen  K,  Lober  WB,  Painter  I.  An  open  source  environment  for  the  statistical  evaluation  of  outbreak  detection 
methods.  AM  I A  Annu  Symp  Proc.  2005;:  1037.  (task  4) 

•  Revere  D,  Madhavan  A,  Kimball  AM,  Turner  A,  Bugni  P,  Fuller  S.  myPublicHealth:  Research  in  Public  Health  Knowledge 
Management  to  Support  Evidence-Based  Practice.  In  Proceedings  of  the  CDC's  Public  Health  Information  Network  (PHIN) 
Conference,  Sept  2006.  Atlanta  GA.  (all  tasks ) 


Associated  Activities: 

•  (2004)  Paladin  Data  Systems  received  a  $1.8  million  contract  from  the  US  Army  Space  and  Missile  Defense  Command.  This 
funding  is  being  used  to  conduct  additional  research  and  expand  the  PSIDTS.  (task  3) 


•  (2006)-  Paladin  Data  Systems  received  notice  of  award  for  $1,882,321  funding  request  for  the  expansion  of  technical  system 

availability;  development  and  validation  of  additional  statistical  techniques  to  improve  the  effectiveness  of  technical  system  in 
support  of  community  health  objectives  and  force  protection/force  readiness  functionality;  implementation  for,  and  training  of 
additional  local  health  jurisdictions  and  military  commands  on  effective  use  of  the  system;  and  extend  the  health  system  inter¬ 
connective  networking  capabilities  developed  for  the  system  in  support  of  the  National  Health  Information  Infrastructure  and 
National  Bio-surveillance  Integration  System  efforts,  (all  tasks ) 
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•  ODIN  project  activities  have  substantially  strengthened  the  University  of  Washington’s  expertise  in  public  health  informatics, 
which  has  lead  to  further  extramural  funding  in  support  of  education,  research,  and  regional  public  health  practice:  (all  tasks) 

o  Center  for  Excellence  in  Public  Health  Informatics 

o  Public  Health  Informatics  Fellowship  Program  in  collaboration  with  the  University  of  Washington  program  in 
Biomedical  and  Health  Informatics  (RWJ/NLM) 

o  Public  Health  Reporting  of  Electronic  Data  (PHRED)  research  support  (WA  Department  of  Health) 
o  Poison  Control  Center  surveillance  support 
o  Fogerty  Grant  for  International  Public  Health  Informatics 


Conclusions 


The  difficulties  in  the  development  of  public  health  informatics  systems  that  impact  multiple  jurisdictions  are  primarily  organizational 
and  political  in  nature.  Progress  on  both  the  technical  development  of  a  system  and  the  organizational  development  can  proceed  in 
parallel  if  sufficient  flexibility  is  built  into  the  technical  architecture.  Further  research  into  public  health  informatics,  especially  in 
regard  to  cross  jurisdictional  issues,  would  be  beneficial. 
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Appendix  A 


Puget  Sound  Infectious  Disease  Tracking  System 


Executive  Management  Team  of  Key  Stakeholders  including  Investigators,  Key 
Participants  and  Administrative  Resources: 


Dr.  Peter  Dunbar,  Principal  Investigator,  Puget  Sound  Infectious  Disease  Tracking 
System 

Dr.  Mark  W.  Oberle  Professor  and  Associate  Dean,  School  of  Public  Health, 

University  of  Washington 

Gary  Macy  Executive  Vice  President/CTO,  Paladin  Data  Systems 
Andy  Fallat  CEO,  Foundation  for  Health  Care  Quality 

Dr.  Ian  Painter,  Biostatistician,  Puget  Sound  Infectious  Disease  Tracking  System 

Jerry  Tonkavich  Consultant,  OTB  Solutions  Group  Seattle,  Washington 

Nigel  Turner,  MPH,  RS  Epidemiologist,  Pierce  County 

Michael  C.  Davisson,  State  of  Washington  Department  of  Health 

Linda  Lekness,  MBA,  MSN,  RN,  Executive  Director,  Puget  Sound  Infectious  Disease 
Tracking  System 

Jude  Van  Buren,  Dr.  PH,  MPH,  RN,  RS  Assistant  Secretary,  Epidemiology,  Health 
Statistics  and  Public  Health  Labs,  State  of  Washington  Department  of  Health 

Dr.  Chris  Leininger,  Chief  Information  Officer,  Swedish  Hospital  Seattle,  Washington 

Dr.  Jeff  Duchin  Chief  Epidemiologist,  Seattle/King  County  Health  Department 

Dr.  Scott  Lindquist,  MD,  MPH  Chief  Epidemiologist,  Seattle/King  County  Health 
Department 
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General  Notes  About  the  Proj  ect  Charter 


Charter  Purpose 

The  project  Charter  is  the  formal  reference  for  shared  assumptions  intended  to  guide 
and  define  a  specific  period  in  the  ODIN  initiative;  providing  a  clear  statement  of  scope 
in  unambiguous,  high-level  terms.  The  project  Charter  is  intended  to  document  shared 
perspectives  on  specific  essential  details  of  why  the  project  is  important,  how  the  project 
will  be  conducted,  how  decisions  will  be  made,  any  known  circumstances  contributing  to 
non-participation  and  other  limiting  factors  and  constraints.  In  addition,  the  Charter 
provides  a  high-level  baseline  of  the  project  timeline  and  budget  and  identifies  major 
events,  milestones,  the  project’s  end  deliverables  and  stakeholder  contributions  to 
achieve  them. 

The  Charter,  in  addition  to  the  detailed  project  plan,  will  be  referred  to  throughout  the 
project  as  a  reference  for  stakeholders  to  initial  project  parameters,  as  a  tool  to  support 
negotiation  and  agreement  on  any  changes  in  scope  or  objectives  and,  overall,  to 
assure  that  goals  for  this  defined  period  of  the  project  are  met.  In  essence,  the  signed 
Charter  will  serve  as  an  agreement  among  ODIN  stakeholders  that  indicates  what  is  to 
be  accomplished  and  who  will  do  it.  The  Charter  provides  formal  recognition  of  the 
project  and  signifies  leadership  authorization  by  collaborating  organizations  for  the 
designated  project  team  to  move  the  project  forward  on  a  defined  path.  As  an  important 
project  reference,  this  Charter  should  be  periodically  updated  as  material  changes  arise. 

Operating  Principles  for  Charter  Development 

1.  This  Charter  has  been  developed  using  a  voluntary  process  through  which  active 
participants  determine  the  consensus  terms  of  engagement  in  the  ODIN  Project. 
The  Charter  Development  Group  (CDG)  includes  designated  participants  from 
organizations  comprising  the  current  ODIN  Steering  Committee.  Organizations 
participating  in  the  charter  development  process  were  permitted  to  withdraw  at  any 
time,  at  which  point  it  was  understood  that  the  organization  would  no  longer  have 
representation  on  the  Project  Steering  Committee.  Future  participation  would  be 
addressed  under  the  direction  of  the  standing  Steering  Committee. 

2.  The  group  made  decisions  by  consensus.  Consensus  is  defined  as  “consent  to 
support”,  and  not  to  oppose,  a  group  decision.  The  group  will  jointly  design  next 
steps  and  ways  to  test  disagreements. 

3.  The  group  decision-making  process  was  driven  by  the  clear  articulation  of  interests 
by  CDG  members.  Members  worked  to  develop  a  common,  clear  understanding  of 
others’  rationale  for  a  given  approach. 

4.  CDG  members  have  been  encouraged  to  test  assumptions.  Group  members  have 
explained  the  reasoning  and  intent  behind  statements,  questions,  and  actions  and 
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have  been  encouraged  to  use  specific  examples  to  reach  agreement  on  what 
important  words  mean. 

5.  The  CDG  included  persons  with  decision-making  authority.  In  cases  where 
designees  participated  on  behalf  of  named  Group  members,  these  designees  have 
maintained  responsibility  for  communicating  all  decisions  and  outcomes  and 
managing  any  required  organizational  response. 

6.  CDG  member  time  has  been  respected  and  every  effort  has  been  made  to  optimize 
the  use  of  time  and  organizational  resources  according  to  the  agreed  upon  Charter 
Development  Approach.  A  special  effort  was  made  to  appropriately  refine  the  scope 
of  the  Charter  to  address  only  critical  elements  of  Steering  Committee  agreement. 

7.  Between  four  in-person  decision-making  meetings,  email  was  the  primary  vehicle  for 
notification  and  communication  among  participants.  Individual  phone  meetings  and 
short  teleconferences  were  used  intermittently  to  facilitate  group  progress.  Going 
forward,  a  combination  of  teleconferences  and  face-to-face  meetings  will  be 
conducted  to  support  progress. 

8.  CDG  members  agreed  to  a  baseline  statement  of  public  health  user  technical  needs 
from  which  more  detailed  requirements  are  being  incrementally  defined.  Future 
requirements,  such  as  those  relating  to  cost-effectiveness  and  sustainability,  will  be 
specified  separately,  as  appropriate. 

9.  During  the  Charter  Development  process  and  going  forward,  CDG  members  will  be 
encouraged  to  share  all  relevant  information  that  could  potentially  impact  the 
success  of  the  project  or  their  continued  participation  in  it. 
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1.0  Executive  Summary 


1.1  Project  Inception 

The  ODIN  Health  Data  Surveillance  and  Analysis  Toolkit  project  (a.k.a.  Outbreak 
Detection  Information  Network)  was  conceived  to  enhance  public  health 
preparedness  and  response  capabilities.  To  date,  the  project  has  been  funded  by 
the  Department  of  Defense  to  help  evaluate  the  effectiveness  of  combining  civilian 
and  military  health  indicator  information  in  order  to  detect  and  respond  to  both 
bioterrorism  events  and  other  critical  health  trends  such  as  avian  flu  and  SARS. 
One  of  the  major  goals  of  the  project  is  to  produce  a  product  that  will  have 
extensive  functionality  for  the  public  health  community  and  provide  a  long  term 
toolset  for  other  public  health  epidemiological  activities. 

ODIN’s  development  occurs  in  a  parallel  timeframe  with  ongoing  implementation  of 
the  Department  of  Defense’s  Electronic  Surveillance  System  for  the  Early 
Notification  of  Community-based  Epidemics  (ESSENCE)  system.  This  system  and 
other  syndromic  surveillance  applications  have  been  implemented  by  the 
Washington  State  Department  of  Health  (DOH)  as  well  as  other  local  health 
jurisdictions  (LHJs),  several  which  are  participants  in  the  ODIN  pilot 
implementation.  Therefore,  another  important  goal  of  the  project  is  to  evaluate 
ODIN’s  added  value  to  state  and  local  surveillance  and  analysis  capability  and 
capacity. 

The  ODIN  Project  was  initially  proposed  by  Paladin  Data  Systems  and  funded  by 
DOD  in  early  2003  as  a  research  initiative.  Additional  funding  was  awarded  to 
continue  the  project  with  current  funding  available  through  December  2007. 
Seattle-based  Foundation  for  Health  Care  Quality,  a  501  (c)3  health  information 
collaborative,  has  provided  public-private  sector  facilitation  among  Paladin,  DOH, 
and  participating  LHJs. 

1 .2  Project  Purpose  and  Description 

At  this  stage,  the  project  is  focused  on  pilot  implementation  and  evaluation  of  the 
ODIN  Health  Data  Surveillance  and  Analysis  Toolkit  (Toolkit)  in  public  health 
settings.  The  Toolkit  will  provide  a  set  of  tools  for  responsible  organizations  to 
assist  in  the  gathering,  analysis,  and  response  to  syndromic  and  other  health 
surveillance  information.  In  addition  to  syndromic  reporting  and  analysis,  the 
envisioned  Toolkit  implementation  will  offer  the  capability  to  receive,  process,  and 
forward  Washington  State  Department  of  Health  (DOH)  reportable  information, 
thereby  allowing  various  information  providers  to  take  advantage  of  the  time  and 
cost  savings  afforded  by  having  a  single  point  of  information  exchange  available 
for  numerous  reporting  requirement  types. 

The  ODIN  Toolkit  will  offer  a  variety  of  analysis  tools  and  visualization  techniques 
allowing  for  complex  statistical,  spatial,  and  temporal  analysis  and  display  of  the 
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information  within  the  system.  The  Toolkit  will  not  only  support  a  number  of 
analysis  tools  “out  of  the  box”,  but  provides  the  framework  for  users  to  develop 
their  own  tools  and  extensions,  and  to  share  those  extensions  with  others  in  the 
project  community. 

Above  all,  this  Toolkit  will  offer  state  and  local  public  health  agencies  a  region-wide 
view  of  the  information  in  the  system,  integrating  and  displaying  data  without 
limitations  that  may  otherwise  be  imposed  by  political  or  organizational  lines. 
Further,  Washington’s  unique  geography  and  concentration  of  DOD  facilities  and 
personnel  require  a  view  not  only  across  county  lines,  but  also  inside  and  outside 
the  gates  of  military  facilities.  This  region-wide  view  will  improve  the  quality  of 
health  care  management  for  all  participating  organizations. 

1.3  Participants 


The  participating  organizations  represented  on  the  Steering  Committee  for  this 
pilot  implementation  and  evaluation  project  and  their  respective  roles  include: 


Participating  Organization 

Key  Project  Role/s 

Washington  State  Department  of  Health 

Pilot  Hosting  Entity; 

Infrastructure/Technical 

Management 

Kitsap  County  Health  District 

LHJ  Data  Sharing  Partner 
(pilot) 

Tacoma  |  Pierce  County  Health  Department 

LHJ  Data  Sharing  Partner 
(pilot) 

Seattle-King  County  Department  of  Public 

Health 

LHJ  Interested  Observer  and 
Ad  Hoc  Advisor 

Paladin  Data  Systems 

DOD  Contractor;  Toolkit 
Developer;  Technical  Project 
Manager 

Foundation  for  Health  Care  Quality 

Subcontractor;  Community 
Facilitator 

1 .4  Objectives 

The  agreed  upon  approach  for  this  unique  project  is  to  pursue  technical 
development  of  the  ODIN  Toolkit  in  parallel  with  a  limited  pilot  implementation  and 
subsequent  evaluation  by  public  health  participants.  Specific  objectives  to  be 
accomplished  in  the  relevant  period  of  this  Charter  include: 


1 .  Take  immediate  steps  to  implement  the  current  version  (v2)  of  the  ODIN 
application  and  evaluate  its  effectiveness  as  a  tool  to  enable  local  health 
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jurisdictions  to  achieve  a  regional  view  of  data  about  public  health  activity 
and  events. 

2.  Beginning  no  later  than  December  1 , 2006,  define  a  category  of  influenza¬ 
like  illness  (ILI)  to  be  monitored  using  ODIN  from  which  to  evaluate  the 
benefit  of  the  Toolkit  in  public  health  practice. 

1 .5  Scope  of  the  Charter 

The  scope  of  this  Charter  pertains  to  activities  required  for  participating  parties  to 
conduct  the  ODIN  Toolkit  pilot  implementation  and  evaluation  occurring  during  the 
DOD-funded  period  from  September  1,  2006  through  December  31,  2007. 

1 .6  Costs/Benefits 

The  Project  Budget  section  offers  a  view  of  budgeted  costs  for  the  period  of  this 
Charter.  Medium  and  long-term  costs,  benefits  and  projected  value  of  a  fully 
implemented  ODIN  Toolkit  will  be  determined  as  an  outcome  of  the  pilot 
experience  and  subsequent  evaluation. 

1 .7  Participant  Expectations 

Project  participants  expect  to  gain  sufficient  understanding  of  the  current  and 
potential  value  of  the  ODIN  Toolkit  relative  to  other  analytical  solutions  used  for 
similar  purposes.  Further,  participants  expect  to  gain  important  insights  into  the 
utility  of  DOH-hosted  surveillance  infrastructure  to  other  LHJs,  both  analytically 
sophisticated  users  and  otherwise,  as  part  of  an  ongoing  effort  to  advance 
statewide  public  health  informatics  capability. 
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2.0  Project  Scope 

1.1  Background 

There  are  several  converging  trends  that  are  reshaping  public  health  practice  and 
effecting  new  requirements  for  health  information.  The  1990’s  ushered  in  a  new 
period  for  communicable  disease  surveillance  and  bioterrorism  preparedness  that 
highlighted  the  need  for  a  more  advanced  electronic  surveillance  infrastructure  that 
takes  advantage  of  current  information  technology  and  supports  integration  and 
streamlining  of  surveillance  practices.  In  1999,  the  Centers  for  Disease  Control  and 
Prevention  (CDC)  proposed  the  development  of  the  National  Electronic  Disease 
Surveillance  System  (NEDSS)  in  partnership  with  public  health  partners  and  state 
and  local  health  departments.  Since  the  anthrax  attacks  in  2001,  the  United  States 
government  has  put  health  information  technology  and  systems  on  a  fast  track  to 
detect,  respond  to  and  contain  the  impact  of  potential  bioterrorism  attacks, 
epidemic/  pandemic  diseases  and  other  large  scale  public  health  events. 
Bioterrorism  funding  increased  dramatically  as  did  efforts  to  improve  public  health’s 
capability  and  capacity  to  detect  and  respond  to  the  threat  of  bioterrorism,  avian  flu 
and  tomorrow’s  pandemics.  In  2002,  the  first  national  Syndromic  Surveillance 
Conference  was  held  in  New  York  City  to  advance  the  science  and  systems  for 
disease  surveillance  across  institutional  and  professional  boundaries.  In  2004,  the 
Public  Health  Information  Network  (PHIN)  received  its  first  funding  to  support  CDC’s 
vision  to  transform  public  health  by  coordinating  its  functions  and  organizations  to 
enable  real-time  data  flow,  computer  assisted  analysis,  decision  support, 
professional  collaboration,  and  rapid  dissemination  of  information  to  public  health, 
the  clinical  care  community  and  the  public.  In  this  same  span  of  time,  the  federal 
government  began  funding  state  and  regional  health  information  infrastructure 
projects  as  part  of  a  broad  vision  for  a  nationwide  health  information  network  to 
improve  quality,  safety  and  accountability  for  health  care.  These  initiatives  require 
significant  advancements  in  technology  and  information  infrastructure  to  effectively 
collect,  analyze  and  interpret  massive  amounts  of  data  about  health  incidents  and 
trends  among  individuals  and  populations. 

The  ODIN  Health  Data  Surveillance  and  Analysis  Toolkit  project  (a.k.a.  Outbreak 
Detection  Information  Network)  was  conceived  to  help  evaluate  the  effectiveness  of 
combining  civilian  health  indicator  information  with  military  health  indicator 
information  in  order  to  detect  and  respond  to  both  bioterrorism  events  and  other 
critical  health  trends  such  as  avian  flu  and  SARS.  ODIN  development  has  been 
funded  by  the  Department  of  Defense,  initially  as  a  research  project.  ODIN  offers 
tools  to  support  analysis  of  very  large  databases  including  the  emerging  science  of 
syndromic  surveillance,  which  uses  the  information  gathered  in  initial  health 
encounters  such  as  emergency  room  visits  in  order  to  predict  disease  outbreaks 
days  to  weeks  ahead  of  clinical  certainty  from  laboratory  results.  Syndromic 
surveillance  is  not  yet  an  accepted  scientific  method,  and  one  of  the  purposes  of 
the  ODIN  project  is  provide  additional  information  and  experience  to  evaluate  it 
relative  to  a  wide  spectrum  of  methodologies.  ODIN  is  also  unique  in  the  fact  that  it 
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is  not  geo-politically  bound,  but  is  designed  with  the  capability  to  provide  visibility 
across  geographic  boundaries.  Thus,  one  of  the  major  goals  of  the  project  is  to 
produce  a  product  that  will  have  extensive  functionality  for  the  public  health 
community  and  provide  a  long  term  toolset  for  other  public  health  epidemiological 
activities. 

ODIN’s  development  occurs  in  parallel  to  ongoing  implementation  of  DOD’s 
Electronic  Surveillance  System  for  the  Early  Notification  of  Community-based 
Epidemics  (ESSENCE)  system.  This  system  and  other  syndromic  surveillance 
applications  have  been  implemented  by  the  ODIN  stakeholder  group  including 
Washington  State  Department  of  Health  as  well  as  several  local  health  jurisdictions 
in  the  state.  Therefore,  another  important  goal  of  the  project  is  to  demonstrate 
ODIN’s  added  value  to  state  and  local  surveillance  and  analysis  objectives. 

Technically,  the  first  key  component  of  ODIN  is  a  dynamic  data  import  engine.  In 
order  for  ODIN  to  operate,  it  requires  data  from  a  wide  variety  of  sources.  It  is 
important  in  the  design  of  ODIN  that  information  providers  be  able  to  supply 
information  in  the  format  and  method  that  is  easiest  for  them,  with  the  ODIN  toolkit 
providing  the  flexibility  to  adapt  to  various  input  streams.  ODIN  is  therefore 
designed  to  accept  data  in  virtually  any  format  by  using  advanced  information 
processing,  analysis  and  quality  control  techniques.  The  second  technical 
component  of  the  ODIN  toolkit  is  a  secure  data  store.  The  secure  data  store  is  at 
the  heart  of  ODIN’s  ability  to  use  statistically  analyzed  information  from  the  past  to 
detect  and  warn  of  anomalous  trends  in  health  indicator  data.  While  one  of  the 
primary  functions  of  ODIN  is  to  generate  alerts  from  current  trends  in  health 
indicator  data,  its  ability  to  analyze  different  periods  in  time  make  it  an  extremely 
valuable  tool  in  analyzing  disease  outbreak  information  from  the  past.  ODIN  will 
provide  a  number  of  user  interfaces,  flexible  display  features,  temporal  mapping 
capabilities,  alerting  and  notification  capabilities,  data  security  and  other 
subcomponents  to  support  overall  functionality.  ODIN’s  flexibility  is  an  important 
attribute  to  its  inherent  value  to  public  health  so  its  purposes  may  be  expanded  as 
needs  are  identified. 

Three  phases  of  development  funding  have  been  received.  As  part  of  the  initial 
funding  for  ODIN,  application  design  was  undertaken  with  varying  levels  of  public 
health  input.  Also,  the  University  of  Washington  received  a  grant  leading  to 
development  of  the  Center  of  Excellence  for  Public  Health  Informatics  which  may 
have  a  future  role  in  related  research.  The  second  phase  of  funding  has  supported 
ODIN  technical  application  development  and  is  nearing  completion.  Today,  a  group 
of  public  health  partners  and  other  stakeholders  have  committed  to  be  firmly 
engaged  in  the  third  phase  of  the  project  based  on  the  agreements  advanced  in  this 
Charter.  Today,  ODIN  has  a  development  funding  source,  a  project  team,  a 
technical  development  team,  and  a  named  group  of  stakeholders  who  have  agreed 
to  implement  and  evaluate  ODIN  in  a  pilot  environment  hosted  by  the  Washington 
State  Department  of  Health. 
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1 .2  Summary  of  Project  Boundaries 

This  pilot  implementation  and  evaluation  project  is  bounded  by  three  major 
elements:  (1)  participating  organizations;  (2)  types  of  information  exchange;  and 
(3)  policy  and  legal  parameters. 

Participating  Organizations 

Six  organizations  comprise  the  project  Steering  Committee  and  serve  in  various 
capacities  as  project  participants.  The  organizational  boundaries  of  this  project 
include  these  entities  and,  to  the  extent  needed,  affiliated  data  sharing  partners  and 
other  stakeholders  as  determined  by  the  Steering  Committee: 

A.  Participants  from  the  State  of  Washington  public  health  system: 

■  Washington  State  Department  of  Health  (hosting  pilot  infrastructure) 

■  Local  Health  Jurisdictions  (LHJs) 

■  Kitsap  County  Health  District  (pilot  participant  /  data  sharing  partner) 

■  Tacoma  |  Pierce  County  Health  Department  (pilot  participant  /  data 
sharing  partner) 

■  Seattle-King  County  Department  of  Public  Health  (LHJ  advisor) 

B.  Paladin  Data  Systems  (systems  development  and  technical  services  vendor, 
contractor  to  DOD) 

C.  Foundation  for  Health  Care  Quality  (project  sponsor,  facilitator) 

Types  of  Information  Exchange 

The  collection  and  analyses  of  varied  sets  of  health  indicator  data  and  subsequent 
exchange  and  use  of  information  by  authorized  entities  is  a  critical  element  at  the 
core  of  this  project.  In  an  effort  to  define  shared  assumptions  about  user  needs  to 
accomplish  these  activities,  the  Steering  Committee  has  agreed  on  a  Statement  of 
User  Needs  and  a  User  Needs  Fulfillment  Plan  intended  to  identify  and  prioritize 
Toolkit  functions  and  features  that  may  be  completed  within  the  project  period  with 
the  funding  available.  These  documents  delineate  the  specific  scope  of  participant 
expectations  pertaining  to  types  of  data  to  be  analyzed  and  technical  development 
to  perform  a  range  of  data  analyses  and  information  exchange  functions.  Some  of 
these  needs  will  be  further  defined  as  the  project  progresses  to  assure  a  clear, 
shared  understanding  about  what  can  and  cannot  be  done  with  the  given  timeline, 
budget  and  the  DOD-contracted  statement  of  work  with  Paladin.  These  documents 
are  included  in  this  Charter  as  Appendix  A  (Statement  of  ODIN  User  Needs)  and 
Appendix  B  (ODIN  User  Needs  Fulfillment  Plan). 
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Policy  and  Legal  Parameters 

This  project  exists  in  an  expanding  continuum  of  health  data  exchange  activities 
under  consideration  at  federal,  state  and  local  levels  in  both  the  public  and  private 
sectors.  As  such,  the  policy  and  legal  basis  for  health  data  exchange  is  evolving. 
The  information  collection,  analysis  and  exchange  conducted  in  this  project  will  be 
subject  to  current  and  relevant  laws,  data  sharing  agreements  and  organizational 
policies.  Stakeholders  will  work  together  through  the  ODIN  Steering  Committee 
and  with  their  community  data  sharing  partners  to  identify  circumstances  in  which 
additional  policy  and/or  legal  provisions  may  be  desired  or  required.  The  Steering 
Committee  will  identify  and  discuss  policy  and  legal  issues  as  they  surface  during 
the  ODIN  pilot  implementation  and  evaluation.  This  group  will  confer  with  legal 
counsel  and  policy-making  bodies  as  required. 


1.3  Project  Goals  and  Key  Objectives 

The  ODIN  Steering  Committee  has  come  together  to  implement  a  statewide 
disease  surveillance  and  analysis  toolkit  with  an  initial  focus  on  Washington’s  Puget 
Sound  region.  The  goals  of  this  information  system  project  include: 

a.  Advance  public  health  capability  and  capacity  to  effectively  detect  and 
respond  to  a  meaningful  range  of  public  health  events  using  both  routine 
and  novel  data  sources; 

b.  Assure  that  the  individual  and  collective  needs  of  key  stakeholder 
organizations  are  met  as  they  pertain  to  assuring  the  value  of  the  ODIN 
toolkit  in  public  health  practice; 

c.  Improve  organizational  and  management  functions  required  to  support 
public  health-private  sector  partnering  on  large  information  technology 
projects  and  health  data  sharing  activities. 

Specific  objectives  for  the  next  12  months,  i.e.,  the  period  of  this  Charter,  reflect 
technical  development  of  the  ODIN  Toolkit  in  parallel  with  a  limited  pilot 
implementation.  Evaluation  planning  activities  will  also  be  undertaken  during  this 
time.  Specific  objectives  to  be  accomplished  in  the  period  of  this  Charter  include: 

OBJECTIVE  1.  Take  immediate  steps  to  implement  the  current  version  (v2)  of 
the  ODIN  application  and  evaluate  its  effectiveness  as  a  tool 
to  enable  local  health  jurisdictions  to  achieve  a  regional  view 
of  data  about  public  health  activity  and  events. 

OBJECTIVE  2.  Beginning  no  later  than  December  1,  2006,  define  a  category 
of  influenza-like  illness  (ILI)  to  be  monitored  using  ODIN  from 
which  to  evaluate  the  benefit  of  the  Toolkit  in  public  health 
practice. 


Document  Name 

Author 

Authorized  by 

Date/ 

Page 

ODIN  Charter  v4 

FNL.doc 

Foundation  for  Health 
Care  Quality 

ODIN  Charter  Development 
Group 

January  4,  2007 

9 

Page  40  of  9 1 


Report  Type 

Project  Name 

Project  Charter 

ODIN  Health  Data  Surveillance  and  Analysis  Toolkit 

1.4  Project  Timeframe  and  Key  Deliverables 

1 .4.1  Project  Start  and  End  Points 

The  relevant  timeline  for  this  project  as  defined  in  the  Charter  follows  the 
availability  of  the  third  and  final  phase  of  funding  to  support  ODIN 
development,  testing  and  initial  deployment.  Using  a  set  of  development 
priorities  as  agreed  upon  by  participants,  this  stage  of  the  ODIN  project  will 
proceed  along  a  timeline  that  begins  September  1,  2006  and  ends 
December  31 , 2007. 


The  funded  phases  of  the  overall  ODIN  project  as  defined  in  the  Charter 
and  key  deliverables  for  each  phase  are  as  follows: 

■  Phase  I:  March  1  2004  -  February  28,  2005 

KEY  DELIVERABLES: 

a)  Initial  requirements  definition 

b)  Initial  application  design 

b)  Prototype  module  development,  version  1.0 


Phase  II:  March  1,  2005  -  June  30,  2006 
KEY  DELIVERABLES: 

a)  Refined  requirements,  version  2.0 

b)  Iterative  design  /  application  architecture 

c)  Functional  pilot  development,  version  2.0 

d)  Test  plan 

Phase  III:  July  1 ,  2006  -  December  2007 
KEY  DELIVERABLES: 

a)  Refined  requirements,  version  3.0 

b)  Pilot  implementation  hosted  by  DOH 

c)  Iterative  development  and  unit  testing,  version  3.0 

d)  Final  testing  and  debugging 

e)  Training  plan 

f)  Evaluation  plan 

g)  Production  product 

h)  User  manual/system  documentation 
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1.4.2  Key  Timing  Considerations,  Constraints  and  Interdependencies 

Key  timing  considerations  may  be  summarized  as  follows: 

■  “Phase  III"  funding  for  this  project  as  it  is  currently  defined  ends 
December  31,  2007.  All  development,  testing,  support  and  training  by 
Paladin  Data  Systems  must  be  completed  prior  to  this  time.  At  the 
close  of  Phase  III  funding,  public  health  stakeholders  must  be  able  to 
determine  what  role  the  ODIN  Toolkit  could  have  in  building  public 
health  analytic  capability  and  how  the  Toolkit  may  be  sustained.  The 
transfer  of  any  legal  rights  to  the  Toolkit  and/or  derivative  works  would 
be  defined  through  a  separate  process. 

■  Washington  State  Department  of  Health,  Kitsap  Health  District  and 
Tacoma  |  Pierce  County  Health  Department  have  agreed  to  participate 
in  a  DOH-hosted  pilot  implementation  during  the  Charter  period.  Data 
sharing  during  this  time  is  dependent  on  technical  feasibility, 
appropriate  data  sharing  agreements,  and  resource  availability. 

Further,  evaluation  of  the  Toolkit  during  this  period  is  dependent  on 
sufficient  health  indicator  data  being  made  available  through 
participating  LHJs  and  their  data  sharing  partners. 

■  Seattle-King  County  Department  of  Public  Health  is  unable  to 
participate  in  the  pilot  implementation  described  in  this  Charter  primarily 
due  to  the  need  to  address  other  organizational  priorities.  Agency 
leadership  has  agreed  to  remain  on  the  Steering  Committee,  to 
continue  to  stay  current  on  ODIN  progress,  and  to  contribute 
perspective  on  evaluation  issues  as  requested. 

1.5  Assumptions  and  Constraints 
1.5.1  Assumptions 

■  This  Charter  assumes  a  12-month  timeline  which  includes  developing, 
testing,  and  piloting  ODIN  at  DOH  and  certifying  the  ODIN  Toolkit  for 
production  use.  At  the  end  of  this  project  as  defined  in  the  Charter,  the 
production  Toolkit  may  be  transitioned  to  the  Department  of  Health. 

■  Each  participant  has  different  existing  capabilities  and  may  have 
different  sets  of  needs  that  they  would  like  to  meet  with  this  project.  It  is 
assumed  that  the  degree  to  which  ODIN  does  or  does  not  meet 
stakeholder  needs  provides  direct  insight  into  the  immediate  and  long¬ 
term  value  proposition  for  the  Toolkit. 

■  The  benefit  of  syndromic  surveillance  as  a  detection  method  has  not 
been  scientifically  proven,  but  it  is  hoped  that  the  proposed  system  will 
be  shown  to  be  of  benefit  in  the  event  of  an  outbreak  or  epidemic  and, 
further,  to  support  routine  surveillance  activities  as  feasible. 
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■  The  Toolkit  functionality  will  be  leveraged  to  perform  other  data  analysis 
activities  as  part  of  assessing  its  overall  value  to  public  and  private 
sector  users  and  stakeholders. 

■  Due  to  unique  circumstances  involving  stakeholder  use  of  ESSENCE 
and  other  surveillance  and  analysis  tools,  some  duplication  of  existing 
functionality  is  unavoidable.  Further,  there  may  be  benefits  that  this 
project  can  realize  that  are  beyond  those  of  syndromic  surveillance 
systems  alone,  for  example,  two-way  communications  between  public 
health  and  hospitals,  improvements  in  analysis  of  reportable  conditions, 
among  others.  As  such,  understanding  the  advantages  and 
disadvantages  of  each  infrastructure  component,  separately  and 
together,  will  be  an  important  aspect  of  evaluation. 

1.5.2  Constraints 

■  Paladin  Data  System  is  under  contract  to  DOD  for  a  specific  statement 
of  work  at  a  defined  funding  level.  While  DOD’s  objectives  include 
advancing  public  health  capabilities,  there  may  be  other  contract 
deliverables  and/or  circumstances  that  Paladin  will  be  compelled  to 
address  that  may  or  may  not  be  aligned  with  public  health  priorities. 

■  Continued  participation  in  this  project  requires  public  health 
stakeholders  to  carefully  balance  a  number  of  competing  factors.  As 
such,  conditions  that  should  be  met  to  ensure  public  health  participation 
include:  (1)  Adequate  resources  will  be  made  available;  (2)  information 
exchange  must  be  technically  feasible  and  acceptable  to  LHJs  and 
clinical  data  sharing  partners;  (3)  the  value  and  benefits  of  the  Toolkit 
must  justify  participation;  and  (4)  project  decision-making  must  support 
the  advancement  of  public  health  interests.  The  perceived  imbalance  of 
any  of  these  conditions  may  be  cause  for  stakeholder  withdrawal. 

1.5.3  Implications 

■  Perhaps  the  single  most  important  implication  inherent  in  the  stated 
assumptions  and  constraints  is  the  need  to  adhere  to  realistic 
expectations  about  project  scope  and  the  level  of  participant 
involvement.  Proactive  project  and  relationship  management  will  be 
critical  to  project  success  during  the  period  of  this  Charter,  and  beyond 
as  applicable. 

■  The  Steering  Committee  understands  that  other  public  health  work  and 
competing  initiatives  will  not  necessarily  remain  stable  during  the 
project  period  and  such  activities  may  impact  public  health  partner 
participation.  As  a  routine  practice,  timely,  effective  communication 
should  be  used  to  facilitate  progress  with  an  emphasis  on  adherence  to 
an  agreed  upon  schedule  to  the  greatest  extent  possible.  However, 
project  managers  would  be  wise  to  anticipate  intermittent  delays  and 
build  some  slack  into  project  timelines. 
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■  Public  health  stakeholders  have  a  range  of  surveillance  systems  and 
capabilities  and  several  have  current  implementations  of  syndromic 
surveillance  systems  and/or  use  DOD’s  ESSENCE  system  supported 
by  the  Johns  Hopkins  University  Applied  Physics  Laboratory. 
Understanding  complimentary  and  potentially  duplicative  functionality 
will  be  an  important  area  of  focus  in  the  evaluation  of  the  Toolkit's 
relative  value  to  public  health.  Public  health  stakeholders  should 
assess  the  Toolkit's  total  contribution  to  public  health  information 
infrastructure  relative  to  the  near-  and  medium-term  cost  to  support  its 
added  value  when  compared  to  comparable  infrastructure  options  at 
state  and  local  levels.  The  demonstration  and  quantification  of  Toolkit 
value  will  be  essential  to  build  a  case  for  future  support  and  broad 
adoption  by  LHJs  and  their  data  sharing  partners  across  the  state. 

■  For  many  reasons,  the  inception,  design  and  development  phases  of 
the  ODIN  initiative  reflect  an  unconventional  approach  to  collaborative 
information  systems  development.  Going  forward,  stakeholders  have 
an  opportunity  to  use  the  testing  and  deployment  phases  of  this  project 
to  advance  their  collective  competence  in  the  practical  application  of 
sound  informatics  principles  in  a  collaborative  environment.  Special 
attention  should  be  given  to  technical  issues,  such  as  data  integration 
and  the  adoption  of  standards;  policy  and  legal  issues,  such  as  data 
sharing  agreements;  and  shared  IT  governance  and  decision-making  to 
advance  cross-programmatic  public  health  information  infrastructure. 
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3.0  Project  Governance  and  Accountability 


3.1  Governance  and  Leadership 

Governance  and  leadership  issues  are  of  critical  importance  to  this  IT  project, 
especially  due  to  the  atypical  nature  of  its  inception,  funding  and  early 
management.  During  the  current  project  timeline  as  defined  in  the  Charter, 
participants  in  the  Steering  Committee  will  have  both  shared  and  distinct  roles  in 
directing  the  project  depending  on  their  chosen  level  of  involvement  in 
implementation  and  evaluation  activities.  All  Steering  Committee  members  will 
contribute  to  the  determination  of  whether  or  not  the  ODIN  Toolkit  adds  sufficient 
value  relative  to  a  larger  public  health  IT  portfolio  to  deploy,  operate  and  maintain 
in  a  production  environment  at  the  close  of  this  project.  Collectively,  the  Steering 
Committee  is  responsible  for  both  short-term,  technical  and  operational  decisions 
coupled  with  longer-term  decisions  regarding  feasibility,  desirability  and 
sustainability. 

Governance  and  leadership  responsibilities  are  focused  on  guiding  the 
stewardship  of  ODIN  in  the  context  of  its  relevance  as  a  vital  public  asset.  That  is, 
as  a  project  paid  for  with  government  funds  intended  to  be  clearly  supportive  of 
protecting  the  health  of  populations,  the  governance  and  leadership  roles  of  the 
ODIN  Steering  Committee  members  named  in  this  Charter  include  both  short-term 
responsibilities  and  longer-term  considerations.  Table  1  describes  specific 
governance  and  leadership  roles  of  the  ODIN  Steering  Committee. 


Table  1.  ODIN  Steering  Committee  Governance  and  Leadership  Roles 


Governance  and  Leadership  Roles 

Accountable  Stakeholders 

Determine  project  feasibility  and  approve 
feasibility  decisions 

■  Department  of  Defense  (budget) 

■  Public  Health  (business  value, 
implementation,  adoption) 

■  Paladin  (technical) 

Define  and  manage  the  project  to  control 
inherent  risks 

■  Foundation  for  Health  Care  Quality 

■  Paladin 

Develop  and  execute  software  development 
and  testing  plans 

■  Paladin 

Review  prototypes  to  validate  Toolkit 
concepts,  architecture  and  functionality  and 
help  inform  how  testing  will  be 
accomplished 

■  Public  Health 
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Governance  and  Leadership  Roles 

Accountable  Stakeholders 

Identify  relevant  policy  and  legal  issues  and 
take  appropriate  action  to  resolve  them 

■  Public  Health 

Determine  how  the  project  recovers  costs 
or  adds  measurable  value  or  positive  cost- 
benefit  to  the  business  (public  health) 
functions  it  is  designed  to  perform 

■  Public  Health 

Inform  the  approach  to  acquiring  data  from 
data  sharing  partners 

■  Public  Health 

Identify  the  need  and  role  for  private 
contractors  under  performance  contracts 

■  Public  Health 

Define  when  independent  experts  should 
assess,  evaluate  and  report  on  various 
aspects  of  the  project  to  stakeholders. 

Such  experts  may  assess,  evaluate  and 
measure: 

■  Technology  solution 

■  Infrastructure  options 

■  Analytics 

■  Data  quality 

■  User  satisfaction 

■  Cost  performance 

■  Other  parameters  as  identified 

■  Public  Health 

Contribute  to  anticipated  cost  projections 
for  maintenance  and  update  of  the  toolkit 

■  Paladin 

Create  a  sound  business  plan  for  the  project 
to  describe  how  it  will  be  sustained 

■  Public  Health 

■  Foundation  for  Health  Care  Quality 

Make  recommendations  for  project  funding 
continuation 

■  Public  Health 

■  Foundation  for  Health  Care  Quality 

3.2  Decision  Making  Structure  and  Process 
Decision-Making  Structure 

Building  on  governance  and  leadership  roles  as  defined  above,  an  agreed  upon 
decision  making  structure  and  processes  will  help  assure  that  each  project 
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stakeholder  has  an  opportunity  to  contemplate  the  consequences  of  new,  proposed 
approaches  or  changes  in  an  existing  project  parameter.  The  Steering  Committee’s 
approach  to  decision-making  encourages  each  stakeholder  to  assess  the  impact  of 
a  new  proposal  or  change  on  his/her  organization,  on  project  outcomes  and  other 
stakeholders,  and  to  constructively  challenge  a  final  decision. 

The  decision-making  practices  adopted  during  the  ODIN  project  timeframe,  as 
defined  in  the  Charter,  take  into  account  the  need  for  continuity  of  leadership  and 
governance  to  achieve  project  objectives  while  also  setting  a  solid  precedent  for 
future  activities.  Figure  1  describes  the  proposed  structure,  which  recognizes 
current  participants  while  also  offering  a  framework  to  support  ongoing 
deliberations,  decision-making  and  evolution  of  the  initiative. 


Figure  1.  ODIN  Project  Decision-Making  Structure 


Decision  Making  Processes 

ODIN  stakeholders  have  agreed  to  the  following  statement  of  objectives  to  help 
drive  the  spirit  and  approach  to  effective  Steering  Committee  decision-making  and 
conflict  management: 

ODIN  Steering  Committee  Decision-Making  Objectives 

The  ODIN  Steering  Committee  agrees: 

■  To  adopt  guidelines  that  support  achievement  of  consensus  on  all  decisions 
put  forth  for  a  vote  by  the  ODIN  Steering  Committee  where  consensus  is 
defined  as  “consent  to  support",  and  not  to  oppose,  a  group  decision. 

■  To  implement  a  decision-making  process  that  accommodates  and  considers 
differences  in  Steering  Committee  member  opinions.  Through  this  process, 
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Steering  Committee  members  will  focus  on  articulating  their  interests  rather 
than  be  bound  by  their  positions;  members  will  balance  advocacy  for 
specific  results  with  inquiry  to  clearly  understand  the  rationale  for  alternative 
approaches. 

■  To  establish  a  decision-making  precedent  for  future  governance  activities  of 
the  ODIN  Steering  Committee. 

In  support  of  achieving  decision-making  objectives,  the  following  guidelines  have 
been  agreed  upon  by  the  ODIN  Steering  Committee  and  will  be  used  to  foster  a 
logical,  orderly  approach  to  decision-making  and  conflict  management. 

ODIN  Steering  Committee  Decision-Making  Guidelines 

1.  Reasonable  effort  will  be  made  to  achieve  a  consensus  decision  among 
the  group  without  use  of  a  voting  process.  Consensus  will  be  achieved 
when  all  organizations  indicate  they  feel  that  discussion  has  been  fully 
carried  out  and  they  are  willing  to  support  a  proposed  decision. 

2.  A  vote  may  be  called  for  when  this  appears  necessary  to  move  the  process 
along  without  undue  loss  of  time. 

3.  No  decision  will  be  voted  on  prematurely,  that  is,  the  group  will  revisit  and 
adjudicate  decisions  as  long  as  productive  debate  and  discussions 
continue. 

4.  At  the  discretion  of  the  group,  agreed  upon  criteria  and  decision-making 
methods  will  be  applied  to  support  specific  decisions. 

5.  Each  participating  organization  will  have  one  vote  to  be  cast  by  an 
authorized  representative  or  their  designee. 

6.  The  core  voting  group  will  be  identified.  Membership  by  any  person 
representing  an  organization  is  voluntary;  an  organization  and/or  individual 
may  choose  to  formally  disengage  from  the  group  for  any  reason  and  at 
such  time  will  not  be  included  in  subsequent  decision-making  processes. 

7.  A  voting  quorum,  either  physically  present  or  participating  by  phone  or  by 
proxy,  is  a  simple  majority  of  the  voting  organizations. 

8.  The  group  may  decide  to  postpone  an  important  vote  if  participation  is  felt 
to  be  too  low. 

9.  A  2/3  majority  vote  of  the  organizations  present  or  participating  by  phone  or 
by  proxy  is  required  to  approve  group  decisions. 

10.  Votes  will  be  conducted  by  voice  vote  of  group  members  in  attendance.  A 
roll-call  vote  can  be  requested  by  any  voting  member  if  the  voice  vote  is 
considered  close. 

1 1 .  Votes  on  important  matters  will  be  announced  in  advance  of  a  meeting. 
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12.  Proxy  voting  is  permitted.  Proxy  votes  may  be  delivered  in  person  by  a 
designee  or  in  writing  (email  is  acceptable)  prior  to  the  vote.  Proxy  votes 
are  allowed  only  for  specific  motions  that  are  announced  in  advance  of  a 
meeting. 

13.  The  group  will  vote  on  decisions  that  are  grounded  in  well-formulated 
issues  and  supported  by  an  appropriate  interval  of  time  to  review  materials, 
identify,  discuss  and  clarify  relevant  matters. 

14.  The  group  may  require  budget  analyses  to  determine  the  potential  impact 
of  decisions  and/or  other  financial  considerations.  In  such  cases,  a  budget 
impact  (or  other)  analysis  must  be  provided  to  inform  decision  making. 

15.  As  needed,  Robert’s  Rules  of  Order  will  be  applied  to  conduct  a  meeting. 

3.3  Responsibilities  for  Issues  Identification  and  Management 

A  shared  issues  identification  and  management  process  will  be  adopted  for 
communicating  relevant  project  issues  to  the  Steering  Committee,  prioritizing  them 
and  effectively  working  toward  successful  resolution.  In  general,  Steering 
Committee  members  or  their  designees  will  represent  their  respective  organizations 
in  issues  identification,  management  and  decision-making  as  follows: 

Public  Health 

The  Washington  State  Department  of  Health  will  be  responsible  for 
identifying  and  communicating  issues  related  to  its  technical  infrastructure 
and  all  related  pilot  implementation  issues  that  impose  requirements  on 
participating  LHJs.  In  the  DOH-hosted  pilot  implementation,  LHJ 
participants,  Kitsap  County  Health  District  and  Tacoma  |  Pierce  County 
Department  of  Health,  will  work  with  DOH  to  draw  on  their  respective 
technical  support  personnel  and  epidemiologists  to  address  issues  and 
help  inform  Steering  Committee  decisions  pertaining  to  data  feeds, 
algorithms/code  sets/analyses,  Toolkit  functionality  and  user  training.  All 
public  health  participants  will  also  drive  decisions  pertaining  to 
participation  of  data  sharing  partners,  Toolkit  evaluation  planning  and 
execution,  determination  of  the  Toolkit’s  overall  value  to  public  health,  and 
all  decisions  pertaining  to  sustainability. 

Paladin  Data  Systems 

Paladin’s  role  in  the  decision-making  process  will  be  focused  on 
collaboratively  addressing  technical  issues  pertaining  to  Toolkit 
development,  testing,  training  and  transitioning  ODIN  software  into 
different  processing  environments  as  directed  by  the  Steering  Committee. 
Paladin  will  provide  technical  support  for  the  Toolkit  during  the  pilot 
implementation  period  at  DOH  and  Paladin  will  be  responsible  for 
surfacing  and  communicating  any  questions  it  may  have  to  clarify  Toolkit 
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requirements,  development  and  testing  issues.  Paladin  will  depend  on 
public  health  participants  to  maintain  timely  bilateral  communication 
regarding  support  and  development  needs.  Paladin  will  allow  adequate, 
yet  defined  time  periods  for  public  health  stakeholder  responses. 
Paladin’s  role  in  decision-making  will  end  at  the  close  of  this  project 
period,  as  defined  in  the  Charter,  or  another  time  as  determined  by  the 
Steering  Committee. 

Foundation  for  Health  Care  Quality 

In  its  project  management  role,  the  Foundation  for  Health  Care  Quality  will 
be  responsible  for  decisions  regarding  coordination  and  communication 
among  project  stakeholders.  The  Steering  Committee  will  advise  the 
Foundation  on  communication  with  external  entities  as  appropriate.  In  this 
role,  the  Foundation  will  also  help  inform  the  Steering  Committee  of  issues 
that  impact  project  timing,  budget  and/or  achievement  of  stated  objectives. 

In  its  consultative  role,  the  Foundation  will  provide  support  for  biostatistical 
tasks  such  as  development  of  proposed  definitions  of  influenza-like  illness 
(ILI)  categories  and  the  related  use  of  various  data  types  in  the  Toolkit. 
The  Foundation  will  also  provide  support  for  user  training  activities  during 
the  pilot  implementation  and  evaluation  period. 

In  its  facilitative  role,  the  Foundation  will  be  responsible  for  effectively 
communicating  and  facilitating  the  satisfactory  resolution  of  issues  as 
directed  by  the  Steering  Committee.  Should  there  be  an  issue  that  cannot 
be  satisfactorily  resolved  after  reasonable  attempts;  the  Foundation  will  be 
responsible  for  considering  use  of  an  objective  third  party  to  help  negotiate 
a  workable  solution.  Should  any  stakeholder  decide  to  withdraw  its 
participation  from  the  project  for  any  reason,  they  should  report  this 
decision  to  the  Foundation. 

Looking  into  2008,  public  health  stakeholders  will  have  assessed  the  results  of  the 
ODIN  Toolkit  evaluation  to  help  define  its  value  and  overall  guidelines  for  any 
additional  development,  testing,  and  refinement  needed.  The  Steering  Committee 
will  then  be  in  a  position  to  make  decisions  about  encouraging  expanded 
participation  in  a  production  environment  and  future  infrastructure  requirements  to 
accommodate  the  core  technology  and  how  it  is  operated,  maintained  and  funded. 
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4.0  Project  Resources 


4.1  Project  Roles 

The  project  includes  a  range  of  roles  that  span  public  and  private  sector 
participants.  Current  primary  project  roles  are  described  in  Table  2  below. 


Table  2.  Primary  Project  Roles 


Title 

Primary  Role  and  Activities 

Stakeholder 

Funding  Agency 

Provide  funding  to  support  project 
activities  as  defined  in  approved 
statements  of  work. 

■  U.S.  Department  of 
Defense 

Project  Sponsor 

Coordinate  and  facilitate 
collaborative  project  activities  and 
provide  consultative  support  as 
needed.  Assist  in  issues 
identification,  management  and 
resolution. 

■  Foundation  for  Health 
Care  Quality  (FHCQ) 

Project  Steering 
Committee 

Identify  and  manage  project 
issues  and  serve  as  the  project’s 
primary  collaborative  decision¬ 
making  body  representing  key 
stakeholders  and  participants. 

■  Washington  Department 
of  Health  (DOH) 

■  Tacoma  |  Pierce  County 
Health  Department 

■  Kitsap  County  Health 
District 

■  Paladin  Data  Systems 

■  Foundation  for  Health 
Care  Quality  (facilitator) 

■  Seattle-King  County 

Dept,  of  Public  Health 
(advisory) 

Public  Health  Domain 
Experts  and  End  Users 

Inform  the  technical  design  and 
development;  provide  functional 
requirements  to  reflect  sound 
analytic  and  epidemiological 
methods  and  support  public 
health  practice;  contribute  to 
priority  setting;  provide  guidance 
on  relationships  with  health  data 
sharing  partners  and  health  data 
acquisition  strategies;  and 
determine  Toolkit  value  and  the 
feasibility  of  long-term  use. 

Washington  Public  Health 
as  represented  by: 

■  Washington  State 
Department  of  Health 

■  Tacoma  |  Pierce  County 
Health  Department 

■  Kitsap  County  Health 
District 

Biostatistician 

Provides  consultative  support  for 
Toolkit  analytical  functionality. 

■  Foundation  for  Health 
Care  Quality  (staff) 

■  Other  resources  as 
identified 

Document  Name 

Author 

Authorized  by 

Date/ 

Page 

ODIN  Charter  v4 

FNL.doc 

Foundation  for  Health 
Care  Quality 

ODIN  Charter  Development 
Group 

January  4,  2007 

20 

Page  51  of  91 


Report  Type 

Project  Name 

Project  Charter 

ODIN  Health  Data  Surveillance  and  Analysis  Toolkit 

Title 

Primary  Role  and  Activities 

Stakeholder 

Senior  Network 
Administrator 

Review  and  set  up  infrastructure; 
establish  and  test  data  feeds 

■  DOH 

■  Paladin 

Technical  Project 
Manager 

Manage  staff,  reports,  scope, 
deliverables,  schedule,  and 
budget  related  to  the  DOD 
contract  and  technical  build 

■  Paladin 

Senior  Advisor 

Provide  technical  direction  and 
advice  on  feasibility  to  meet 
business  needs 

■  Paladin 

Senior 

Developer/Designer 

Design  and  develop  application 
including  analysis 

■  Paladin 

Developer 

Develop  application  modules  and 
interfaces  including  data  stream 
extractions 

■  Paladin 

Database  Administrator 

Database  set  up,  tuning, 
performance,  and  maintenance  to 
support  the  application 

■  Paladin 

Graphics  Specialist 

Develop  graphical  interface  and 
any  brand  items  such  as  logos, 
etc.  for  the  project 

■  Paladin 

Tester 

Test  system  and  assist  with 
specification  documentation 

■  Paladin 

Trainer 

Develop  training  plan  and  deliver 
training  sessions 

■  DOH 

■  FHCQ 

■  Paladin 

4.2  Public  Health  Executive  Committee 

The  ODIN  Executive  Committee  includes  the  following  Steering  Committee 
members  that  represent  their  respective  public  health  organization  and  have 
the  authority  to  make  decisions  on  its  behalf. 

■  Jude  Van  Buren:  Assistant  Secretary,  Washington  State  Department  of 
Health 

■  Dr.  Vic  Harris,  Deputy  Director,  Tacoma  |  Pierce  County 

■  Dr.  Scott  Lindquist,  Director  of  Health,  Kitsap  Health  District 
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Executive  Committee  roles  and  responsibilities  include: 

A.  Contribute  to  the  success  of  the  project,  in  which  they  have  a  vested 
interest; 

B.  Serve  as  key  decision  makers  on  critical  issues  including  those 
pertaining  to  decisions  with  strategic,  operational  and  budget 
implications; 

C.  Support  Work  Group  requests  for  assistance  on  critical  issues;  and 

D.  Provide  guidance  and  supervision  of  Work  Group  members  from  their 
respective  organizations  as  needed  to  support  achievement  of  project 
objectives. 

4.3  Paladin  Data  Systems 

Serving  as  the  technology  contractor  for  this  project,  Paladin  will  be 
represented  on  the  Steering  Committee  by  Gary  Macy,  Chief  Technology 
Officer. 

4.4  Foundation  for  Health  Care  Quality 

During  the  timeline  described  in  this  Charter,  the  collaborative  aspects  of  the 
ODIN  project  are  being  supported  by  the  Foundation  for  Health  Care  Quality. 
Foundation  roles  and  responsibilities  include  providing  a  collaborative 
framework  and  environment  in  which  to  convene,  facilitate  and  otherwise 
support  the  overall  achievement  of  project  objectives.  Foundation  staff  and 
assigned  contractors  will  manage  the  collaborative  aspects  of  the  project, 
including  but  not  limited  to,  convening  meetings,  producing  meeting  minutes, 
providing  biostatistical  domain  expertise  and  exploring  specific  technical  and 
non-technical  issues  as  required  or  requested  by  project  participants.  In  this 
role,  the  Foundation  will  be  represented  on  the  Steering  Committee.  The 
following  project  leaders,  managers  and  staff  will  support  the  project  as  it  is 
currently  defined: 

■  Andy  Fallat,  Project  Executive  Sponsor  and  CEO,  Foundation  for  Health 
Care  Quality 

■  Peter  Dunbar,  MD,  ODIN  Project  Principal  Investigator 

■  Linda  Lekness,  ODIN  Project  Manager,  Foundation  for  Health  Care 
Quality 

■  Ian  Painter,  PhD,  Biostatistician,  Project  Staff,  Foundation  for  Health 
Care  Quality 
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4.5  Working  Groups 

Working  groups  include  participant  personnel  grouped  according  to  their 
specific  expertise  to  accomplish  defined  aspects  of  the  project.  Today,  these 
working  groups  are  loosely  organized  around  technical  issues  and  public 
health  practice  issues.  Participation  will  be  adjusted  as  needed  to 
accomplish  project  goals.  Current  working  group  participants  are  listed. 

Technical  Development,  Testing  and  Deployment  Work  Group: 

■  Phill  Lowe,  Epidemiology  Data  Systems  Manager,  Informatics  Program, 
Washington  State  Department  of  Health 

■  Michael  Davisson,  IT  Director,  Public  Health  Labs  /  WEDSS  Project 
Director,  Washington  State  Department  of  Health 

■  Scott  Horn,  Information  Services,  Kitsap  Health  District 

■  Anne  Harvey,  Technical  Project  Manager,  Paladin  Data  Systems 

■  Ian  Painter,  PhD,  Biostatistician,  Foundation  for  Health  Care  Quality 

■  LHJ  technical  support  as  assigned 

■  Paladin  developers  and  analysts  as  assigned 

Public  Health  Domain  Work  Group: 

■  Jo  Hofmann,  MD,  State  Communicable  Disease  Epidemiologist  and  CD 
Office  Director,  Washington  State  Department  of  Health 

■  Judith  May,  Bioterrorism  Surveillance  and  Epidemiology  Manager, 
Communicable  Disease  Epidemiology,  Washington  State  Department  of 
Health 

■  Donna  Duffy,  Epidemiology  Surveillance  Coordinator  for  DOH 
Bioterrorism,  Washington  State  Department  of  Health 

■  Eva  Crim,  Health  Information  Resource  Program  Manager,  Kitsap  Health 
District 

■  Nicola  Marsden-Haug,  Epidemiologist  II,  Washington  State  Project 
Coordinator 

■  Nigel  Turner,  Public  Health  Manager,  Communicable  Disease  Control, 
Tacoma  -  Pierce  County 
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4.6  Contractor  Resources 

Contractor  resources  will  be  engaged  from  time  to  time  as  required  to 
support  specific  project  needs.  While  some  work  group  staff  are  contract 
personnel  to  stakeholder  organizations,  this  resource  category  is  primarily 
reserved  for  contractors  engaged  to  support  specific  aspects  of  the  project. 
The  Foundation  for  Health  Care  Quality  contracts  with  the  following 
personnel  in  a  consultative  capacity  and  to  assist  in  this  project: 

■  Laura  Ripp,  Charter  editor 

Other  contractors  will  be  defined  by  the  Steering  Committee. 


4.7  Other  IT  Personnel  and  Resources 

This  category  of  resources  will  be  used  to  describe  additional  personnel  that 
may  contribute  to  the  project  in  less  structured  roles.  Currently,  one  person 
is  providing  facilitation  assistance  to  the  project  during  the  Charter 
development  process  and  may  potentially  serve  in  an  ongoing  role: 

■  Bud  Nicola,  MD,  MHSA,  FACPM,  Director,  Community-Oriented  Public 
Health  Practice  Program  Faculty,  Northwest  Center  for  Public  Health 
Practice  Senior  Consultant  and  CDC  Assignee  University  of  Washington 
School  of  Public  Health  and  Community  Medicine  Department  of  Health 
Services 


Document  Name 

Author 

Authorized  by 

Date/ 

Page 

ODIN  Charter  v4 

FNL.doc 

Foundation  for  Health 
Care  Quality 

ODIN  Charter  Development 
Group 

January  4,  2007 

24 

Page  55  of  91 


Report  Type 

Project  Name 

Project  Charter 

ODIN  Health  Data  Surveillance  and  Analysis  Toolkit 

5.0  Evaluation 

The  ODIN  Steering  Committee  agrees  that  evaluation  of  the  Toolkit  is  essential  to  gain 
insight  into  its  actual  value  to  public  health  and  the  likelihood  of  statewide  adoption  and 
use.  The  information  and  experience  gained  through  initial  evaluation  activities  will  be 
used  in  part  to  develop  a  long  term  plan  for  ongoing  evaluation  of  the  Toolkit,  including 
an  approach  to  update  coders  and  algorithms.  Details  of  a  formal  evaluation  plan  are  to 
be  developed  during  this  Charter  period  with  DOH  taking  the  lead  on  plan  design  for 
review,  comment  and  approval  by  the  Steering  Committee.  The  Steering  Committee’s 
guiding  assumptions  for  ODIN  evaluation  include  the  following: 

a.  Evaluation  activities  are  intended  to  represent  the  needs  of  public  health 
agencies  that  will  rely  on  the  ODIN  Toolkit  for  analytical  support  pertaining  to 
numerous  data  sets,  to  be  defined. 

b.  A  detailed  Evaluation  Plan  will  be  developed  to  support  this  Charter  and  the 
overall  ODIN  initiative  and,  at  a  minimum,  will  pertain  to  those  activities  that  are 
feasible  and  desirable  to  implement  during  the  period  ending  December  31, 

2007. 

c.  Evaluation  deliverables,  including  but  not  limited  to  findings  and  implications  for 
system  development  and  refinement,  will  be  shared  with  the  ODIN  Project 
Steering  Committee  and  system  developers. 

d.  Recommended  changes  to  ODIN  will  be  assessed  as  to  their  priority  as  defined 
by  public  health  users  and  managed  according  to  a  formal  change  control 
process  as  defined  by  system  developers. 

e.  Evaluation  will  be  conducted  in  parallel  with  pilot  testing  and  implementation  of 
ODIN,  to  be  hosted  by  the  Washington  State  Department  of  Health  for  use  by  the 
State  and  participating  local  health  jurisdictions  (LHJs). 

f.  The  context  for  evaluation  is  the  integration  of  real-time  data  feeds  across 
independent  organizations  and  agencies  that  may  have  different  surveillance  and 
analytical  systems  in  use.  Further,  ODIN’s  contribution  to  achieving  the  goal  of 
regional  surveillance  should  be  reflected  in  the  evaluation  plan. 

g.  The  funding  and  mechanism/s  for  conducting  evaluation  activities  will  be 
determined  by  the  ODIN  Project  Steering  Committee.  Funding  must  be  derived 
from  existing  project  funds  or  some  other  readily  available,  known  funding  source 
that  can  be  encumbered  to  support  a  defined  evaluation  scope  of  work. 

h.  The  final  version  of  this  Evaluation  Approach  document  will  be  incorporated  into 
the  ODIN  Health  Data  Surveillance  and  Analysis  Toolkit  Project  Charter  as  an 
addendum  representing  the  consensus  agreement  among  the  ODIN  Project 
Steering  Committee. 
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The  ODIN  Toolkit  Evaluation  Plan  will  incorporate  questions  of  interest  to  public  health 
participants  which  include,  but  are  not  limited  to  the  following: 

1.  Can  each  participating  county  (Kitsap  and  Tacoma  |  Pierce)  successfully  send 
relevant  data  types  to  DOH? 

2.  Was  two-way  data  flow  accomplished  between  pilot  locations  and  DOH? 

3.  Was  someone  available  at  each  user  location  to  look  at  the  data? 

4.  Is  staff  adequately  trained  to  understand  the  data? 

5.  Does  the  ODIN  Toolkit  give  public  health  users  a  view  of  the  data  that  is  new  and 
useful? 

6.  Did  each  participating  county  draft  a  response  plan? 

7.  Can  public  health  use  the  Toolkit  to  create  insight  into  certain  issues  and  events 
that  actually  change/improve  public  health  response? 

8.  How  technically  robust  is  the  Toolkit? 

9.  Does  the  ODIN  Toolkit  give  public  health  added  value?  If  so,  is  the  value  added 
adequate  to  justify  ongoing  support? 
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6.0  Risk  Management 


6.1  Project  Risk  Identification,  Management  and  Mitigation  Plan 

Table  3  describes  current  project  risks,  their  potential  for  occurrence  and  key 
steps  to  mitigate  and  manage  such  risks. 


Table  3.  High-Level  Project  Risk  Assessment 


Major  Project  Risks 

POTENTIAL 

Occurrence 
(H,  M,  L) 

Management  /  Mitigation  Plan 

1 .  Project  Steering  Committee 
does  not  agree  (as  a  whole)  to 
the  project  scope. 

M 

Commit  to  define  user  requirements 
throughout  the  design  and 
development  phases  of  the  project 
and  ensure  project  delivers  a 
solution  that  meets  individual  and 
collective  requirements. 

2.  State  and  county  health 

agencies  have  disparate  needs 
that  may  compromise  the  overall 
project  objectives. 

M 

Define  and  document  each 
participant’s  individual  needs  and 
work  toward  a  consensus  statement 
of  needs  for  a  regional  solution. 

3.  Knowledgeable  staff  leaves  early 
in  the  process  (especially  public 
health  and  technical  staff.) 

M 

Intermittently  reaffirm  participation 
as  required  to  actively  identify  and 
manage  risk  and  to  constructively 
address  required  changes. 

4.  Hospital  relationships  with  local 
public  health  agencies  could  be 
compromised  by  proposed 
changes. 

M 

Work  with  LHJs  to  ensure 
communication  channels  and 
messages  are  established  prior  to 
any  changes  occurring. 

5.  Current  hospital  data  stream 
extractions  cannot  effectively 
accommodate  new  functionality. 

M 

Clearly  define  system  and  business 
requirements  to  allow  for  adequate 
planning,  review,  and  additional 
effort  required  to  involve  hospitals. 

6.  Steering  Committee  members 
may  be  too  busy  to  provide 
timely  decisions  on  project 
efforts. 

H 

Provide  adequate  feedback  time  for 
response  and  comments  and 
identify  the  Steering  Committee  as 
the  ultimate  decision-making  body. 

7.  Resource  constraints  may 
prevent  or  seriously  limit  public 
health  stakeholder  involvement 
in  Toolkit  implementation  and 
evaluation. 

M 

Provide  funding  according  to  an 
equitable,  agreed  upon  project 
budget  allocation  to  support  public 
health  participation  in  Toolkit 
implementation  and  evaluation 
activities. 
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Risk  management  activities  are  intrinsically  anticipatory,  therefore,  as 
appropriate;  the  Steering  Committee  will  consider  future  deployment  decisions 
to  manage  risks  related  to  this  initiative  in  an  effort  to  promote  the  best  possible 
outcomes.  In  anticipation  of  a  scenario  in  which  the  ODIN  Toolkit  has  been 
determined  to  provide  sufficient  added  value  to  public  health  to  warrant 
continued  investment  and  production  deployment,  the  following  risk 
management  practices  will  be  employed: 

■  Review  and  anticipate  implementation  challenges  following  the  pilot  period. 
This  includes  reconciling  the  Toolkit  with  practical  use  (business  process) 
requirements  to  help  assure  the  product  meets  intended  user  objectives  and 
to  promote  adoption. 

■  Clearly  identify  security,  bandwidth  and  internal  systems  interface 
considerations  in  the  pilot  stage. 

■  Evaluate  hardware  performance  in  the  existing  production  environment 
rather  than  rely  on  manufacturer  (or  other)  specifications. 

■  Ensure  that  installed  hardware  meets  or  exceeds  specifications. 

■  Assure  adequate  staff  resources. 

■  Assure  adequate  funding  is  available  to  meet  future  project  needs. 

■  Include  informed  overestimates  for  time  and  costs  to  support 
troubleshooting  and  resolving  unforeseen  technology  issues. 

■  Develop  an  IT  customer  service  agreement  before  implementation. 

■  Ensure  sufficient  training  preparation  time  as  well  as  sufficient  training  time. 

■  Complete  training  activities  according  to  a  well  designed  plan. 

■  Before  full  implementation,  clearly  define  the  transitions  from  the  vendor’s 
project  team  to  the  vendor’s  product  support  help  desk  to  the  final 
deployment  model  including  user  support,  maintenance  and  operations. 

■  Assure  follow-up  and  provide  adequate  user  support  during  the  initial 
phases  of  regional/statewide  rollout. 
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7.0  Communication 


7.1  Communication  Principles 

The  overall  principles  that  will  guide  ODIN  project  communications  include: 

■  Use  the  Project  Charter  to  keep  current  and  reinforce  shared 
agreements  on  project  goals  and  the  strategic  significance  of  the  effort. 

■  Work  collectively  to  increase  understanding  among  all  stakeholders  and 
expose  and  clarify  areas  of  misunderstanding. 

■  Support  tactical  coordination  among  project  managers,  project 
participants,  intended  users,  and  developers. 

■  Regularly  recognize  and  acknowledge  project  milestones. 

■  Express  the  importance  of  stakeholder  groups  by  demonstrating  that 
their  interests  are  being  addressed. 

■  Support  relationship-building  through  timely,  effective,  bidirectional 
communication  with  key  stakeholders  and  other  interested  parties. 


7.2  Approach  to  Internal  and  External  Project  Communication 

Project  communication  includes  both  internal  and  external  dimensions.  The 
approach  to  internal  communication  includes  consideration  of  the 
stakeholders,  their  information  needs,  the  format  and  frequency  of  such 
communications,  and  the  process  that  will  be  most  useful  for  conveying  the 
needed  information.  The  following  communication  activities  will  be  used  to 
ensure  accurate  and  appropriate  information  dissemination  within  the  team: 

■  Project  Charter  approval  and  use  as  a  living  reference  document  to 
reinforce  stakeholder  agreements. 

■  Face-to-face  project  status  and  information-sharing  meetings  as  needed 
to  support  group  cohesion  and  goal  achievement. 

■  Conference  calls  and/or  web  conferences  to  review  project  development 
and  issues. 

■  Communication  content  development  responsibilities  will  be  assigned  by 
the  Steering  Committee.  Such  activities  will  include  development  and 
distribution  of  meeting  minutes,  fact  sheets,  written  status  reports,  and 
project  updates. 

The  approach  to  external  communication  is  grounded  in  achieving 
consistency  and  clarity  in  formal  and  informal  messages  that  may  have 
implications  for  outside  entities.  Strategic  communication  planning  is  an 
important  Steering  Committee  role  especially  in  a  scenario  that  implies 
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possible  expansion  of  ODIN’s  use  by  other  LHJs  and  their  health  data 
sharing  partners  such  as  clinics,  hospitals,  laboratories,  health  systems  and 
health  data  aggregators,  among  others.  Near-term  communication  with 
external  parties  will  be  planned  and  conducted  by  the  Steering  Committee 
as  required.  However,  the  Steering  Committee  agrees  that  a  more  formal 
communication  plan  is  needed  to  manage  structured  external  messages 
over  time.  Therefore,  a  more  formal  plan  will  evolve  according  to  project 
outcomes  and  as  subsequent  communication  needs  arise. 
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8.0  Going  Forward  Project  Plan 


8.1  Project  Approach  and  Work  Plan 

The  ODIN  pilot  implementation  and  evaluation  effort  involves  three  major 
phases  of  work:  (1)  software  development  and  testing;  (2)  evaluation; 
and  (3)  transition.  These  phases,  their  associated  activities  and  the 
estimated  timeline  for  performance  are  represented  in  the  project  work 
plan  in  Figure  2.  The  Steering  Committee  is  responsible  for  all 
refinements  to  the  work  plan  and  will  define  all  specific  meeting  dates  and 
agendas  needed  to  sufficiently  define  project  details  over  time. 

8.2  Project  Management  and  Operations  Accountability 

Due  to  the  nature  of  the  project  relationships  and  its  unique  structure, 
there  are  some  inherent  project  management  challenges  related  to 
accountability  for  required  deliverables.  Paladin  Data  Systems  retains 
accountability  for  satisfactory  completion  of  specific  contract  deliverables 
to  DOD  within  a  defined  period  of  performance.  Paladin’s  responsibility 
includes  detailed  project  management  functions  as  they  pertain  to 
software  development,  testing,  deployment  and  technical  documentation. 

Public  health  participants  are  not  held  to  the  same  level  of  contractual 
accountability  as  Paladin.  However,  they  have  agreed  to  engage  in  the 
project  activities  described  in  this  Charter  and  to  make  their  best  effort  to 
be  responsive  to  the  timeline  and  requests  for  input  where  public  health 
expertise  is  needed.  As  suggested  in  the  work  plan  and  throughout  this 
Charter,  Paladin  will  work  closely  with  public  health  participants  to 
implement  the  pilot  environment  and  address  refinements  to  finalize 
production-level  Toolkit  requirements  as  mutually  agreed  upon  and  as 
feasible  within  the  project  budget,  scope  and  timeline.  It  should  be  noted 
that  Public  health  participants  have  assumed  responsibility  for  Toolkit 
evaluation  and  will  take  a  leadership  role  in  planning  and  executing  this 
important  element  of  the  project. 

The  Foundation  for  Health  Care  Quality  serves  in  an  important  role  to 
bridge  the  discrepancy  in  accountability  for  completion  of  project 
deliverables  between  Paladin  and  public  health  participants.  The 
Foundation  has  assumed  responsibility  for  facilitating  the  clear 
articulation  of  commitments  and  evolving  agreements  among  the  parties 
for  the  duration  of  the  Charter  period.  The  Foundation  is  also  responsible 
for  critical  collaborative  project  management  functions  including 
maintaining  good  communication  among  project  stakeholders,  convening 
meetings,  conducting  teleconferences,  documenting  action  steps  and 
decisions,  and  performing  all  necessary  follow-up  to  propel  the  project 
forward  according  to  shared  assumptions. 
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Figure  2.  ODIN  Work  Plan  and  Estimated  Timeline 

ODIN  Health  Data  Surveillance  &  Analysis  Toolkit 
ODIN  Toolkit  Pilot  Implementation  and  Evaluation  Timeline-September  2006  -  December  2007 
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SOFTWARE  DEVELOPMENT  and  TESTING 


ODIN  V. 2  available  to  stakeholders  for  pilot  at  DOH 


Initial  user  training  for  pilot  implementation  (DOH/FHCQ) 


Define  ILI  categories  and  data  requirements  (FHCQ/LHJs/DOH) 


Establish  data  feeds;  perform  V. 2  testing  (Paladin/DOH/LHJs) 


Incorporate  additional  requirements  into  V.3  development  cycle 


V.3  development  and  testing  iteration  (Paladin) 


V.3  development  cycle  ends;  formal  change  request  required 


V.3  full  unit  testing,  final  testing  and  debugging  (Paladin) 


V.3  production  version  available  (Paladin 


V.3  Maintenance  and  Minor  Changes  (Paladin) 


System  Certification  for  Production  Use  (Paladin) 


Final  Training  Plan  Development  (Paladin) 


User  Manual  /  Toolkit  Documentation  Development  (Paladin) 


V.3  Finalized  and  Published  (Paladin) 


Transition  production  Toolkit  to  DOH  (Paladin/DOH) 


EVALUATION 


Develop  and  refine  the  Evaluation  Plan  (DOH  lead) 


Finalize  and  engage  evaluation  team 


Implement  Evaluation  Plan  (roles  TBD) 


Deliver  Interim  and  Final  Evaluation  Reports 


TRANSITION 


Develop  formal  Communication  Plan  as  required 


Implement  external  Communication  Plan  if  applicable 


System  transitions  to  WA  DOH  if  applicable 


♦  :  Key  Milestones  and  Steering  Committee  Decision  Points 


^=Activity  continues  beyond  project  timeline 
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Appendix  A:  Statement  of  ODI N  User  Needs 
ODIN  Health  Data  Surveillance  and  Analysis  Toolkit 

Approved  by  ODI  N  Steering  Committee  February  16,  2006 


1.  Public  Health  and  Partners’  Current  User  Needs  (note:  letters  in  [braces]  signify 

cross  references  to  the  list  in  Appendix  B:  ODIN  User  Needs  Fulfillment  Plan.) 

User  Interface 

a.  [q]  Provide  a  single,  common  look-and-feel  user  console  for  the  toolkit.  All  LHJ, 
DOH  and  data  provider  users  will  use  the  same  tools  for  performing  updates, 
data  and  statistical  algorithm  displays. 

b.  [r,  b]  Provide  the  data  providers  with  the  same  analysis  capability  for  their  data  as 
LHJs  and  DOH  have  including  access  to  their  analyzed  data  via  secure  web  sites 
(updated  on  an  automated,  daily  basis  or  more  frequently  if  necessary.) 

c.  [a]  Provide  a  Washington  state  regional  analysis  view: 

i.  DOH  and  LHJs  will  be  able  to  see  regional  analyses;  and 

ii.  LHJs  will  not  have  access  to  other  LHJs  data  analyses  unless  explicitly 
agreed  upon. 

Access  Controls 

d.  [c]  Include  role-based  security  with  roles  for  data  providers,  LHJs  and  DOH. 

e.  [d]  Include  security  protections  such  that  one  data  provider  cannot  see  another 
data  provider’s  data,  and  one  LHJ  cannot  see  another  LHJ’s  data. 

Analysis,  Visualization  and  Reporting 

f.  [i]  Include  current  functionality  used  by  LHJs  as  a  part  of  the  ODIN  toolkit. 

g.  [I]  Include  the  capability  to  maintain  an  “unlimited”  number  of  unique  syndromic 
analysis  algorithms  and  syndromic  condition  coding  algorithms: 

i.  [h]  Include  the  best  and  most  current  chief  complaint  coders  and  data 
analysis  algorithms  as  defined  by  technical  and  academic  experts  in 
advanced  surveillance  methods. 

ii.  [e,  g]  Provide  dynamic  capability  for  multiple  syndromic  coders  (including, 
but  not  limited  to,  NYC,  RODS,  EARS,  ESSENCE)  along  with  the  ability 
for  individual  users  to  create,  upload,  modify,  maintain  and  store 
customized  coding  algorithms. 

iii.  0,  k]  Include  the  capability  for  LHJ  and  DOH  users  to  easily  duplicate,  add, 
update,  modify  syndromic  statistical  analysis  algorithms  for  event 
detection  analysis. 
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h.  [p]  Include  user-specific  analysis  capabilities;  this  would  include  simple  statistics 
and  trend  analysis  on  data. 

i.  [m]  Include  the  capability  to  perform  syndromic  statistical  analysis  by  both  data 
provider  location  &  patient  residence  and  to  map  the  output  of  these  analyses. 

j.  [n]  Include  the  capability  for  analyses  to  be  run  at  preset  periods  several  times 
per  day  and  on  demand  so  that  results  can  be  refreshed  if  data  are  received  from 
data  providers  at  different  times  throughout  the  day. 

k.  [v]  Include  automated  capability  to  continuously  evaluate  data  quality  and 
promptly  address  problems  with  data,  (e.g.,  missing  or  incomplete  data).  Users 
will  have  the  ability  to  designate  IT  personnel  at  each  data  provider  who  will 
receive  automated  notifications  when  missing  data  need  to  be  resubmitted. 

l.  [w]  Include  the  ability  to  handle  different  identifiers  and  de-identifiers,  with  a  goal 
towards  standardized  de-identifiers  and  identifiers. 

m.  [u]  Include  the  ability  to  download  the  data  using  common  data  formats  (e.g.,  .xls, 
.txt,  .dbf,  xml). 

n.  [f]  Include  the  ability  to  add  additional  data  sources  to  the  toolkit  (e.g.,  school 
absenteeism,  pharmacy  sales,  laboratory  results)  and  develop  tools  for  analyzing 
those  data  sources  (See  #2). 

Communication  and  Notification 

o.  [s]  Include  automated  notifications  of  statistical  anomalies  in  the  data,  with  an 
ability  to  configure  the  notification  by  LHJ  (this  includes  who  in  the  LHJ  is  notified 
and  what  constitutes  a  statistical  anomaly).  The  users  will  select  which 
algorithms  to  monitor  for  notification.  Should  be  provided  in  the  context  of  DOH 
"SECURES"  communication  system. 

p.  [x]  Provide  tools  for  ODIN  users  to  facilitate  communication  with  one  another  to 
discuss  aberrations  detected  by  the  toolkit.  For  example,  users  will  have  the 
ability  to  flag  a  signal  that  requires  follow-up  or  might  be  of  interest  to  other 
users,  and  to  post  a  message  next  to  that  item  as  well  as  to  distribute  email  to  a 
selection  of  users  who  should  review  that  signal. 

RESEARCH  AND  EVALUATION 

q.  [o]  Evaluate  existing  data  sources,  algorithms,  and  coders  concurrently  with 
toolkit  development  to  ensure  that  the  tools  available  in  the  toolkit  are  relevant 
and  are  meeting  user  needs. 

r.  [t]  Include  the  ability  to  perform  data  extracts  for  research  according  to  applicable 
data  sharing  provisions  (with  written  approval  of  each  LHJ,  as  some  existing  data 
sharing  agreements  with  data  providers  may  explicitly  forbid  use  of  data  for  non- 
Public  Health  purposes.) 
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2.  Additional  Data  Sources  for  Potential  Linkage  to  the  Toolkit  for  Analysis 

a.  Poison  Control  Data 

b.  Coded  Emergency  Department  discharge  information 

c.  Over  the  Counter  Pharmacy  sales  information 

d.  Lab  data  both  from  Data  providers  and  commercial  laboratories  including  the 
feeds  from  Electronic  Laboratory  Reporting  Systems  and  aggregator  systems 
such  as  PHRED 

i.  Laboratory  reportable  conditions 

ii.  Select  Laboratory  Test  Requests 

e.  Communicable  Disease  case  reports 

f.  Primary  care  discharge  information 

g.  RODS  -  pharmacy  sales  information 

h.  BioSense 

i.  PHIMS  reports 

j.  School  and  business  absenteeism  data 

k.  Medical  examiner  data 

l.  EMS/911  dispatch  data 

m.  Other  data  sources  as  deemed  acceptable  by  Charter  Members 
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Appendix  B:  ODI N  User  Needs  Fulfi llment  Plan 


1  ODIN  Health  Data  Surveillance  &  Analysis 
Toolkit:  User  Needs 

ODIN  Current  and  Planned  Development  Status 

||  USER  PRIORITIES  (l=higher  4=lower)  1 1 

DOH 

SKC 

Kitsap 

Tac- 

Pierce 

COMBINED  II 

PH  USER 
Priority  (Est.)| 

FUNCTIONALITY  CURRENTLY  INCLUDED  IN  ODIN  VERSION  2 

la.  Provide  a  WA  slate  regional  analysis  view  (see  subitems  (lai)  and 
(laii) 

rhis  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 
April  demonstration  CD-ROM's  and  will  be  available  in  the  July  1  Version  2  Release 

1 

1 

1 

1 

1 

lai.  DOH  and  LHJs  will  be  able  to  see  regional  analyses 

rhis  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 

April  demonstration  CD-ROM’s  and  will  be  available  in  the  July  1  Version  2  Release 

2 

1 

1 

1 

1 

laii.  LHJs  will  not  have  access  to  other  LHJs  data  analyses. 

This  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 

April  demonstration  CD-ROM's  and  will  be  available  in  the  July  1  Version  2  Release 

1 

1 

1 

1 

1 

1b.  Provide  the  data  providers  with  access  to  their  analyzed  data  via 

secure  web  sites  (updated  on  an  automated,  daily  basis  or  more 
frequently  if  necessary.) 

rhis  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 

April  demonstration  CD-ROM’s  and  will  be  available  in  the  July  1  Version  2  Release 

1 

1 

1 

1 

1 

1c.  Include  role-based  security  with  roles  for  data  providers.  LHJs 

and  DOH. 

rhis  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 
April  demonstration  CD-ROM’s  and  will  be  available  in  the  July  1  Version  2  Release 

1 

1 

1 

1 

1 

Id.  Include  security  protections  such  that  one  hospital  cannot  see 

another  hospital's  data,  and  one  LHJ  cannot  see  another  LHJ's  data. 

rhis  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 
April  demonstration  CD-ROM's  and  will  be  available  in  the  July  1  Version  2  Release 

1 

1 

1 

1 

1 

1m.  Include  the  capability  to  perform  syndromic  statistical  analysis  by 

both  by  hospital  location  &  patient  residence. 

rhis  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 

April  demonstration  CD-ROM’s  and  will  be  available  in  the  July  1  Version  2  Release 

1 

1 

1 

4 

1 

1q.  Provide  a  single,  common  look-and-feel  user  console  for  the 

toolkit.  All  LHJs,  DOH  and  hospital  users  will  use  the  same  tools  for 
performing  updates,  data  and  statistical  algorithm  displays. 

rhis  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 

April  demonstration  CD-ROM's  and  will  be  available  in  the  July  1  Version  2  Release 

1 

1 

1 

4 

1 

1u.  Include  the  ability  to  download  the  data  using  common  data 

formats  (e.g.,  .xls,  .txt,  .dbf.  xml). 

rhis  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 
April  demonstration  CD-ROM's  and  will  be  available  in  the  July  1  Version  2  Release 

1 

1 

1 

1 
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ODIN  Current  and  Planned  Development  Status 


USER  PRIORITIES  (l=higher  4=lower) 


DOH 


SKC 


Kitsap 


Tac- 

Pierce 


COMBINED 

PH  USER 


FUNCTIONALITY  CURRENTLY  INCLUDED  IN  ODIN  VERSION  2 

1v.  Include  automated  capability  to  continuously  evaluate  data  and 
promptly  address  problems  with  data.  /  USER  NOTES:  This  item  is 
currently  available  in  our  own  system  as  well  as  in  the  ESSENCE 
system,  and  1  would  argue  that  it  is  important  that  this  feature  be 
available  immediately. 

This  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 
April  demonstration  CD-ROM's  and  will  be  available  in  the  July  1  Version  2  Release 

1 

1 

1 

1 

i 

1e.  Provide  dynamic  capability  for  multiple  syndromic  coders 

(including,  but  not  limited  to.  NYC,  RODS,  EARS,  ESSENCE)  along 
with  the  ability  for  individual  LHJs  to  create,  modify,  and  store 
customized  coding  algorithms.  /  USER  NOTES:  Users  must  be  able 
to  create  and  modify  the  algorithms  on-the-fly. 

This  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 

April  demonstration  CD-ROM's  and  will  be  available  in  the  July  1  Version  2  Release. 

ODIN  was  designed  to  be  expansible  and  modifiable.  In  order  to  actually  make 
modifications,  technical  computer  skills  are  required  to  link  into  the  data  structures,  and 
code  the  mathematical  algorithms  to  perform  complex  statistical  analysis  in  a  compatible 
toolset. 

2 

1 

1 

4 

1-2 

1g.  Include  the  ability  to  create,  upload  and  maintain  the  chief 

complaint  coding  algorithms  by  adding,  editing,  and  deleting 
additional  local  codes  and  algorithms  dynamically.  /  USER  NOTES: 
Similar  to  1e. 

This  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 
April  demonstration  CD-ROM's  and  will  be  available  in  the  July  1  Version  2  Release. 

ODIN  was  designed  to  be  expansible  and  modifiable.  In  order  to  actually  make 
modifications,  technical  computer  skills  are  required  to  link  into  the  data  structures,  and 
code  the  mathematical  algorithms  to  perform  complex  statistical  analysis  in  a  compatible 
toolset.  A  responsible  system  manager  (probably  the  State  DOH  hosting  agency)  would 
control  exactly  what  is  added  and  modified  in  the  core  system,  while  individual  LHJ’s 
could  perform  extracts  and  local  analysis  for  more  esoteric  studies 

2 

1 

1 

1 

1 

1h.  Include  the  best  and  most  current  chief  complaint  coders  and 

data  analysis  algorithms  as  defined  by  technical  and  academic 
experts  in  advanced  surveillance  methods.  1  USER  NOTES:  Coders 
and  algorithms  must  be  current  to  make  the  analytical  output  useful. 

This  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 

April  demonstration  CD-ROM's  and  will  be  available  in  the  July  1  Version  2  Release.  As 
this  requirement  is  constantly  changing  as  research  in  the  field  of  syndromic  surveillance 
progresses,  it  will  have  to  be  upgraded  periodically. 

2 

1 

1 

1 

1-2 

1o.  Evaluate  existing  data  sources,  algorithms,  and  coders 

concurrently.  /  USER  NOTES:  This  is  one  aspect  of  ODIN  that  could 
serve  some  benefit  to  LHJs  as  we  do  not  currently  have  the  resources 
to  evaluate  the  named  elements. 

This  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 
April  demonstration  CD-ROM's  and  will  be  available  in  the  July  1  Version  2  Release 

1 

1 

1 

1 

1 

If.  Include  the  ability  to  add  additional  data  sources  to  the  toolkit 

(e.g.,  school  absenteeism,  pharmacy  sales,  laboratory  results)  and 
develop  tools  for  analyzing  those  data  sources. 

The  functionality  to  meet  this  requirement  is  inherent  in  the  basic  design  of  the  ODIN 
system.  ODIN  was  designed  to  be  expansible  and  modifiable.  While  there  is  no  system 
imposed  limit  to  the  number  of  data  types  and  analysis  tools  that  can  be  supported  there 
are  practical  limits  related  to  the  "time  to  process"  and  computational  intensity  of  the 
operations.  It  should  be  noted  that  some  level  of  computer  and  information  management 
skills  will  be  required  to  accomplish  these  tasks.  A  responsible  system  manager 
(probably  the  State  DOH  hosting  agency)  would  control  exactly  what  is  added  and 
modified  in  the  core  system. 

2 

1 

4 

3 
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ODIN  Current  and  Planned  Development  Status 


FUNCTIONALITY  CURRENTLY  INCLUDED  IN  ODIN  VERSION  2 


1  USER  PRIORITIES  (l=higher  4=lower)  | 

DOH 

SKC 

Kitsap 

Tac- 

Pierce 

COMBINED 

PH  USER 
Priority  (Est.) 

1j.  Include  the  ability  to  easily  modify,  duplicate  or  update  standard 
syndromic  statistical  analysis  algorithms.  /  USER  NOTES:  Assume 
this  the  same  or  very  similar  to  1e. 

rhe  functionality  to  meet  this  requirement  is  inherent  in  the  basic  design  of  the  ODIN 
system.  ODIN  was  designed  to  be  expansible  and  modifiable.  That  being  said,  the 
concept  of  "easily”  is  a  very  relative  term.  In  order  to  actually  make  modifications, 
technical  computer  skills  are  required  to  link  into  the  data  structures,  and  code  the 
mathematical  algorithms  to  perform  complex  statistical  analysis  in  a  compatible  toolset. 

A  responsible  system  manager  (probably  the  State  DOH  hosting  agency)  would  control 
exactly  what  is  added  and  modified  in  the  core  system,  while  individual  LHJs  could 
perform  extracts  and  local  analysis  for  more  esoteric  studies 

1 

1 

1 

4 

1 

Ik.  Include  the  capability  for  LHJ  and  DOH  users  to  duplicate,  add, 

update,  modify  syndromic  statistical  analysis  algorithms  for  event 
detection  analysis.  /  USER  NOTES:  Similar  to  1j. 

The  functionality  to  meet  this  requirement  is  inherent  in  the  basic  design  of  the  ODIN 

system.  ODIN  was  designed  to  be  expansible  and  modifiable.  That  being  said,  the 
concept  of  "easily"  is  a  very  relative  term.  In  order  to  actually  make  modifications, 
technical  computer  skills  are  required  to  link  into  the  data  structures,  and  code  the 
algorithms  to  provide  the  required  functionality  in  a  compatible  toolset  A  responsible 
system  manager  (probably  the  State  DOH  hosting  agency)  would  control  exactly  what  is 
added  and  modified  in  the  core  system,  while  individual  LHJ's  could  perform  extracts  and 
local  analysis  for  more  esoteric  studies 

1 

1 

1 

4 

1 

11.  Include  the  capability  to  maintain  an  'Unlimited  number  of  unique 

syndromic  analysis  algorithms  and  syndromic  condition  coding 
algorithms.  /  USER  NOTES:  Could  fold  the  word  "unlimited'  into  1j 

Lhe  functionality  to  meet  this  requirement  is  inherent  in  the  basic  design  of  the  ODIN 

system.  ODIN  was  designed  to  be  expansible  and  modifiable.  While  there  is  no  physical 
limit  to  the  number  of  analysis  and  coding  systems  that  can  be  supported  there  are 
practical  limits  related  to  the  "time  to  process"  and  computational  intensity  of  the 
operations.  A  responsible  system  manager  (probably  the  State  DOH  hosting  agency) 
would  control  exactly  what  is  added  and  modified  in  the  core  system,  while  individual 

LHJ's  could  perform  extracts  and  local  analysis  for  more  esoteric  studies 

1 

1 

1 

4 

1 

In.  Include  the  capability  for  analyses  to  be  run  at  preset  periods 

several  times  per  day  and  on  demand  so  that  results  can  be  refreshed 
if  data  are  received  from  data  providers  at  different  times  throughout 
the  day.  1  USER  NOTES:  LHJs  will  have  an  immediate  need  to 
refresh  the  data  on  demand. 

Data  is  analyzed  ”on-the-fly"  as  users  request  vanous  display  from  the  system.  Currently, 

a  separate  import  process  accepts,  aggregates,  and  performs  some  statistical  processing 
at  a  fixed  time,  once  a  day.  This  process  takes  several  minutes  for  an  entire  state’s  worth 
of  data,  and  system  response  for  users  will  be  reduced  while  the  process  is  running. 
Currently  the  system  does  not  contain  logic  to  handle  partial  updates  to  an  individual 
facility's  information,  and  errors  would  result  from  a  single  day's  data  being  transmitted  in 
two  pieces.  Analysis  needs  to  be  done  to  more  fully  understand  how  partial  data  feeds 
would  be  sent  to  ensure  that  the  import  processes  can  be  modified  to  handle  the  various 
permutations  (full  or  partial  re-transmittal,  correction,  deletions,  additions)  of  data 
availability  accurately. 

1 

1 

1 

2 

1 
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1  ODIN  Health  Data  Surveillance  &  Analysis 
Toolkit:  User  Needs 

ODIN  Current  and  Planned  Development  Status 

1  USER  PRIORITIES  (l=higher  4=lower)  |  j 

DOH 

SKC 

Kitsap 

Tac- 

Pierce 

COMBINED  I 

PH  USER 
Priority  (Est.)| 

FUNCTIONALITY  CURRENTLY  INCLUDED  IN  ODIN  VERSION  2 

1w.  Include  the  ability  to  handle  different  identifiers  and  de-identifiers, 
with  a  goal  towards  standardized  de-identifiers  and  identifiers. 

ODIN  accepts  and  stores,  but  does  not  process,  coded  identification  information.  While  it 
is  possible  to  include  a  decoding  capability,  this  functionality  may  not  practical  from  a 
policies  and  rules  basis.  At  this  time  users  can  see  the  coded  identification  information 
that  was  transmitted  with  any  particular  detail  record  that  they  have  valid  access  to,  and 
can  then  contact  the  supplying  organization  to  identify  the  specific  individual.  More 
definition  of  this  requirement  is  required  to  determine  how  it  might  affect  the  future 
versions  of  the  system. 

3 

3 

1 

2 

2 

2g.  RODS  -  pharmacy  sales  information 

This  functionality  is  in  the  Version  2  release  of  ODIN  (using  simulated  RODS  data)  which 

will  be  previewed  on  the  mid-April  demonstration  CD-ROM's  and  will  be  available  in  the 

July  1  Version  2  Release 

2 

2 

4 

3 

2 

1r.  Provide  the  data  providers  with  the  same  analysis  capability  for 

their  data  as  LHJs  and  DOH  have. 

rhis  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 

April  demonstration  CD-ROM's  and  will  be  available  in  the  July  1  Version  2  Release 

4 

3 

1 

4 

3 

It.  Include  the  ability  to  perform  data  extracts  for  research  (with 

written  approval  of  each  LHJ.  as  some  existing  data  sharing 
agreements  with  data  providers  may  explicitly  forbid  use  of  data  for 
non-Public  Health  purposes.) 

This  functionality  is  in  the  Version  2  release  of  ODIN  which  will  be  previewed  on  the  mid- 

April  demonstration  CD-ROM's  and  will  be  available  in  the  July  1  Version  2  Release 

4 

3 

4 

3 

3-4 
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Report  Type 

Project  Name 

Project  Charter 

ODIN  Health  Data  Surveillance  and  Analysis  Toolkit 

ODIN  Health  Data  Surveillance  &  Analysis 
Toolkit:  User  Needs 


ODIN  Current  and  Planned  Development  Status 


USER  PRIORITIES  (l=higher  4=lower) 


DOH 


SKC 


Kitsap 


Tac- 

Pierce 


COMBINED 
PH  USER 
Priority  (Est.) 


FUNCTIONALITY  PARTIALLY  DEVELOPED  IN  ODIN  V2-ADDITIONAL  REQUIREMENTS  NEEDED 


1i.  Include  current  functionality  used  by  LHJs  as  a  part  of  the  ODIN 
toolkit.  /  USER  NOTES:  Goal  is  to  avoid  referring  to  multiple,  slightly 
different  systems.  It  will  be  essential  for  ODIN  tools  to  be  evaluated 
(especially  relative  to  ESSENCE)  prior  to  their  release. 

It  is  believed  that  ODIN  will  encompass  the  requested  functionality,  but  not  in  an  identical 
manner  to  the  ESSENCE  system,  however  this  belief  cannot  be  validated  until  the  V2 
system  has  been  reviewed  by  the  stakeholders,  and  any  important  deficiencies  stacked 
into  the  V3  development  plan. 

1 

1 

1 

4 

1-2 

Ip.  Include  LHJ-specific  analysis  capabilities;  this  would  include 

simple  statistics  and  trend  analysis  on  data.  /  USER  NOTES:  Similar 
to  1e 

This  functionality  is  partially  developed  in  V2,  but  requires  interactive  dialog  with  the 

stakeholders  leading  to  a  comprehensive  user  requirements  understanding  to  be  fully 
implemented 

1 

1 

1 

2 

1 

2d.  Lab  data  both  from  data  providers  and  commercial  laboratories 

including  the  feeds  from  Electronic  Laboratory  Reporting  Systems  and 
aggregator  systems  such  as  PHRED 

The  framework  to  support  this  functionality  exists  in  V2.  but  it  is  not  implemented  for  this 
specific  information  type  yet.  This  is  a  dimension  that  might  be  included  in  V3.  Two 
factors  will  play  into  the  actual  implementation  plan.  The  primary  factor  will  be  the 
success  of  locating  and  establishing  a  relationship  with  a  suitable  source  for  the  data. 

The  second  would  be  the  relative  priority  of  this  specific  data  feed  when  considered  with 
the  other  novel  data  feeds  that  are  available  for  implementation. 

1 

2 

4 

3 

2 

2di.  Laboratory  reportable  conditions 

The  framework  to  support  this  functionality  exists  in  V2,  but  it  is  not  implemented  for  this 
specific  information  type  yet.  This  is  a  dimension  that  might  be  included  in  V3.  Two 
factors  will  play  into  the  actual  implementation  plan.  The  primary  factor  will  be  the 
success  of  locating  and  establishing  a  relationship  with  a  suitable  source  for  the  data. 

The  second  would  be  the  relative  priority  of  this  specific  data  feed  when  considered  with 
the  other  novel  data  feeds  that  are  available  for  implementation. 

1 

2 

4 

3 

2 

2j.  School  and  business  absenteeism  data  /  USER  NOTES:  Some 

LHJs  will  be  receiving  these  data  shortly  and  we  want  to  have  the 
ability  to  analyze  them  immediately. 

The  framework  to  support  this  functionality  exists  in  V2,  but  it  is  not  implemented  for  this 

specific  information  type  yet.  This  is  a  dimension  that  might  be  included  in  V3.  Two 
factors  will  play  into  the  actual  implementation  plan.  The  primary  factor  will  be  the 
success  of  locating  and  establishing  a  relationship  with  a  suitable  source  for  the  data. 

The  second  would  be  the  relative  priority  of  this  specific  data  feed  when  considered  with 
the  other  novel  data  feeds  that  are  available  for  implementation. 

2 

1 

4 

4 

2 
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Report  Type 

Project  Name 

Project  Charter 

ODIN  Health  Data  Surveillance  and  Analysis  Toolkit 

1  ODIN  Health  Data  Surveillance  &  Analysis 
Toolkit:  User  Needs 

ODIN  Current  and  Planned  Development  Status 

|  USER  PRIORITIES  (l=higher  4=lower)  || 

DOH 

SKC 

Kitsap 

Tac- 

Pierce 

COMBINED 

PH  USER 
Priority  (Est.) 

FUNCTIONALITY  PARTIALLY  DEVELOPED  IN  ODIN  V2-ADDITIONAL  REQUIREMENTS  NEEDED 

21.  EMS/91 1  dispatch  data  /  USER  NOTES:  Many  systems  currently 
have  these  data  and  it  would  be  helpful  if  LHJs  could  analyze  these 
data  immediately. 

The  framework  to  support  this  functionality  exists  in  V 2,  but  it  is  not  implemented  for  this 
specific  information  type  yet.  This  is  a  dimension  that  might  be  included  in  V3.  Two 
factors  will  play  into  the  actual  implementation  plan.  The  primary  factor  will  be  the 
success  of  locating  and  establishing  a  relationship  with  a  suitable  source  for  the  data. 

The  second  would  be  the  relative  priority  of  this  specific  data  feed  when  considered  with 
the  other  novel  data  feeds  that  are  available  for  implementation. 

3 

1 

4 

4 

2 

2a.  Poison  Control  Data  /  USER  NOTES:  Some  LHJs  should  be 

receiving  these  data  shortly  and  we  want  to  have  the  ability  to  analyze 
them  immediately. 

The  framework  to  support  this  functionality  exists  in  V2,  but  it  is  not  implemented  for  this 
specific  information  type  yet.  This  is  a  dimension  that  might  be  included  in  V3.  Two 
factors  will  play  into  the  actual  implementation  plan.  The  primary  factor  will  be  the 
success  of  locating  and  establishing  a  relationship  with  a  suitable  source  for  the  data. 

The  second  would  be  the  relative  priority  of  this  specific  data  feed  when  considered  with 
the  other  novel  data  feeds  that  are  available  for  implementation. 

1 

1 

4 

1 

1 

2e.  Communicable  Disease  case  reports 

The  framework  to  support  this  functionality  exists  in  V2,  but  it  is  not  implemented  for  this 

specific  information  type  yet.  This  is  a  dimension  that  might  be  included  in  V3.  Two 
factors  will  play  into  the  actual  implementation  plan.  The  primary  factor  will  be  the 
success  of  locating  and  establishing  a  relationship  with  a  suitable  source  for  the  data. 

The  second  would  be  the  relative  priority  of  this  specific  data  feed  when  considered  with 
the  other  novel  data  feeds  that  are  available  for  implementation. 

2 

3 

4 

2 

2-3 

2h.  BioSense 

It  is  believed  that  ODIN  will  encompass  most  BioSense  functionality,  but  not  in  an 

identical  manner  to  the  BioSense  system,  however  this  belief  cannot  be  validated  until  the 
V2  system  has  been  reviewed  by  the  stakeholders,  and  any  important  deficiencies 
stacked  into  the  V3  development  plan. 

2 

3 

4 

3 

2-3 

2c.  Over  the  Counter  Pharmacy  sales  information 

fhe  framework  to  support  this  functionality  exists  in  V2,  but  it  is  not  implemented  for  this 
specific  information  type  yet.  This  is  a  dimension  that  might  be  included  in  V3.  Two 
factors  will  play  into  the  actual  implementation  plan.  The  primary  factor  will  be  the 
success  of  locating  and  establishing  a  relationship  with  a  suitable  source  for  the  data. 

The  second  would  be  the  relative  priority  of  this  specific  data  feed  when  considered  with 
the  other  novel  data  feeds  that  are  available  for  implementation. 

3 

2 

4 

3 

3 
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Report  Type 

Project  Name 

Project  Charter 

ODIN  Health  Data  Surveillance  and  Analysis  Toolkit 

1  ODIN  Health  Data  Surveillance  &  Analysis 
Toolkit:  User  Needs 

ODIN  Current  and  Planned  Development  Status 

1  USER  PRIORITIES  (l=higher  4=lower)  |  J 

|  DOH 

SKC 

Kitsap 

Tac- 

Pierce 

COMBINED  I 

PH  USER 
Priority  (Est.)l 

FUNCTIONALITY  PARTIALLY  DEVELOPED  IN  ODIN  V2-ADDITIONAL  REQUIREMENTS  NEEDED 

2dii.  Select  Laboratory  Test  Requests 

The  framework  to  support  this  functionality  exists  in  V2,  but  it  is  not  implemented  for  this 
specific  information  type  yet.  This  is  a  dimension  that  might  be  included  in  V3.  Two 
factors  will  play  into  the  actual  implementation  plan.  The  primary  factor  will  be  the 
success  of  locating  and  establishing  a  relationship  with  a  suitable  source  for  the  data. 

The  second  would  be  the  relative  priority  of  this  specific  data  feed  when  considered  with 
the  other  novel  data  feeds  that  are  available  for  implementation. 

3 

2 

4 

3 

3 

2f.  Primary  care  discharge  information 

Support  for  this  functionality  exists  in  V2,  but  it  is  not  implemented  for  this  specific 

information  type  yet.  More  definition  of  this  requirement  is  required.  It  is  unclear  if  this 
information  is  available  from  primary  care  providers  in  an  electronic  form.  Does  this  refer 
to  a  diagnostic  disposition  such  as  an  ICD9.  or  to  another  code  referring  to  patient 
disposition  (admitted,  release,  deceased).  Need  to  develop  a  better  definition  of  the 
availability  and  consistency  of  this  data  element  in  the  data  source  community. 

1 

3 

4 

1 

3 

2i.  PHIMS  reports 

rhis  functionality  exists  in  V2.  This  is  a  dimension  that  might  be  included  in  V3.  Two 
factors  will  play  into  the  actual  implementation  plan.  The  primary  factor  will  be  the 
success  of  locating  and  establishing  a  relationship  with  a  suitable  source  for  the  data. 

The  second  would  be  the  relative  priority  of  this  specific  data  feed  when  considered  with 
the  other  novel  data  feeds  that  are  available  for  implementation. 

2 

3 

4 

2 

3 

2K.  Medical  examiner  data 

rhe  framework  to  support  this  functionality  exists  in  V2,  but  it  is  not  implemented  for  this 

specific  information  type  yet.  This  is  a  dimension  that  might  be  included  in  V3.  Two 
factors  will  play  into  the  actual  implementation  plan.  The  primary  factor  will  be  the 
success  of  locating  and  establishing  a  relationship  with  a  suitable  source  for  the  data. 

The  second  would  be  the  relative  priority  of  this  specific  data  feed  when  considered  with 
the  other  novel  data  feeds  that  are  available  for  implementation. 

3 

2 

4 

4 

3-4 
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Report  Type 

Project  Name 

Project  Charter 

ODIN  Health  Data  Surveillance  and  Analysis  Toolkit 

1  ODIN  Health  Data  Surveillance  &  Analysis 
Toolkit:  User  Needs 

ODIN  Current  and  Planned  Development  Status 

|  USER  PRIORITIES  (l=higher  4=lower)  \\ 

1  DOH 

SKC 

Kitsap 

Tac- 

Pierce 

COMBINED  1 

PH  USER 
Priority  (Est.)l 

FUNCTIONALITY  DEFFERED  UNTIL  ODIN  VERSION  3-ADDITIONAL  REQUIREMENTS  NEEDED 

lx.  Provide  tools  for  ODIN  users  to  facilitate  communication  with  one 
another  to  discuss  aberrations  detected  by  the  system.  For  example, 
users  will  have  the  ability  to  flag  a  signal  that  requires  follow-up  or 
might  be  of  interest  to  other  users,  and  to  post  a  message  next  to  that 
item  as  well  as  to  distribute  email  to  a  selection  of  users  who  should 
review  that  signal. 

Development  of  this  functionality  has  been  deferred  until  Version  3  of  ODIN  to  allow  for  an 
interactive  dialog  with  the  stakeholders  leading  to  a  comprehensive  user  requirements 
understanding 

2 

1 

1 

1 

1 

Is.  Include  automated  notifications  of  statistical  anomalies  in  the 

data,  with  an  ability  to  configure  the  notification  by  LHJ  (this  includes 
who  in  the  LHJ  is  notified  and  what  constitutes  a  statistical  anomaly). 
The  users  will  select  which  algorithms  to  monitor  for  notification. 

Should  be  provided  in  the  context  of  DOH  "SECURES" 
communication  system.  USER  NOTES:  Must  closely  monitor  the  use 
of  automated  notifications,  at  least  until  the  algorithms  are  completely 
refined  to  avoid  numerous  false  alerts. 

This  requirement  is  related  to  requirement  lx.  Development  of  this  functionality  has  been 
deferred  until  Version  3  of  ODIN  to  allow  for  an  interactive  dialog  with  the  stakeholders 
leading  to  a  comprehensive  user  requirements  understanding 

2 

3 

1 

1 

2 

ADDITIONAL  REQUIREMENTS  NEEDED  TO  DETERMINE  FEASIBILITY 

■2b.  Coded  Emergency  Department  discharge  information  /  USER 
BnOTES:  This  functionality  is  currently  available  in  some  LHJ  systemsl 
las  well  as  in  ESSENCE,  and  will  be  key  for  pandemic  influenza 
Isurveillance. 

■More  definition  of  this  requirement  is  required.  Does  this  refer  to  a  diagnostic  disposition  I 
Isuch  as  an  ICD9,  or  to  another  code  referring  to  patient  disposition  (admitted,  release, 
[deceased).  Need  to  develop  a  better  definition  of  the  availability  and  consistency  of  this  1 
[data  element  in  the  data  source  community. 

□ 

u 

□ 

□ 

end  appendix  b. 
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Appendix  C 


ODIN  End  User  Needs  From  Charter  Process  and  ODIN  status  at 

start  of  charter  process 

ODIN 

Status 
February  06 

np  =  not  present 
cud  =  currently  under 
development  (i.e., 
started  but  not 
completed) 

Notes 

Note:  Items  preceeded  by  1.  =  Public  Health  and  Partners’  Immediate  User  Requirements 

Note:  Items  preceeded  by  2.  =  Additional  Data  Sources  for  Potential  Linkage  to  the 
System  for  Analysis 

Note:  Rows  are  in  order  of  average  jurisdiction  priority 

la.  Provide  a  WA  state  regional  analysis  view  (see  subitems  (1  ai)  and  (laii) 

present 

laii.  LHJs  will  not  have  access  to  other  LHJs  data  analyses. 

present 

1c.  Include  role-based  security  with  roles  for  hospitals,  LHJs  and  DOH. 

present 

Id.  Include  security  protections  such  that  one  hospital  cannot  see  another  hospital’s  data,  and 
one  LHJ  cannot  see  another  LHJ’s  data. 

present 

If.  Include  the  ability  to  add  additional  data  sources  to  the  system  (e.g.,  school  absenteeism, 
pharmacy  sales,  laboratory  results)  and  develop  tools  for  analyzing  those  data  sources. 

present 

1m.  Include  the  capability  to  perform  syndromic  statistical  analysis  by  both  by  hospital  location 
&  patient  residence. 

present 

1u.  Include  the  ability  to  download  the  data  using  common  data  formats  (e.g.,  .xls,  .txt,  .dbf, 
xml). 

present 

1v.  Include  automated  capability  to  continuously  evaluate  data  and  promptly  address 
problems  with  data. 

present 

1  ai.  DOH  and  LHJs  will  be  able  to  see  regional  analyses;  and 

cud 

1  b.  Provide  the  hospitals  with  access  to  their  analyzed  data  via  secure  web  sites  (updated  on 
an  automated,  daily  basis  or  more  frequently  if  necessary.) 

cud 
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1e.  Provide  dynamic  capability  for  multiple  syndromic  coders  (including,  but  not  limited  to, 
NYC,  RODS,  EARS,  ESSENCE)  along  with  the  ability  for  individual  LHJs  to  create, 
modify,  and  store  customized  coding  algorithms. 


1  h.  Include  the  best  and  most  current  chief  complaint  coders  and  data  analysis  algorithms  as 
defined  by  technical  and  academic  experts  in  advanced  surveillance  methods. 


1j.  Include  the  ability  to  easily  modify,  duplicate  or  update  standard  syndromic  statistical 
analysis  algorithms. 


Ik.  Include  the  capability  for  LHJ  and  DOH  users  to  duplicate,  add,  update,  modify  syndromic 
statistical  analysis  algorithms  for  event  detection  analysis. 


II.  Include  the  capability  to  maintain  an  “Unlimited”  number  of  unique  syndromic  analysis 
algorithms  and  syndromic  condition  coding  algorithms. 


1o.  Include  the  ability  to  evaluate  existing  data  sources,  algorithms,  and  coders  concurrently. 


Ip.  Include  LHJ-specific  analysis  capabilities;  this  would  include  simple  statistics  and  trend 
analysis  on  data. 


1q.  Provide  a  single,  common  look-and-feel  user  console  for  the  system.  All  LHJs,  DOH  and 
hospital  users  will  use  the  same  tools  for  performing  updates,  data  and  statistical 
algorithm  displays. 


2d.  Lab  data  both  from  Hospitals  and  commercial  laboratories  including  the  feeds  from 

Electronic  Laboratory  Reporting  Systems  and  aggregator  systems  such  as  PHRED 


2di.  Laboratory  reportable  conditions 


2dii.  Select  Laboratory  Test  Requests 


1g.  Include  the  ability  to  create,  upload  and  maintain  the  chief  complaint  coding  algorithms  by 
adding,  editing,  and  deleting  additional  local  codes  and  algorithms  dynamically. 


Is.  Include  automated  notifications  of  statistical  anomalies  in  the  data,  with  an  ability  to 
configure  the  notification  by  LHJ  (this  includes  who  in  the  LHJ  is  notified  and  what 
constitutes  a  statistical  anomaly).  The  users  will  select  which  algorithms  to  monitor  for 
notification,  e.g.,  the  ESSENCE  algorithm. 


2b.  Coded  Emergency  Department  discharge  information 


2g.  RODS  -  pharmacy  sales  information 


2c.  Over  the  Counter  Pharmacy  sales  information 


Requirements 

under 

development 


Through  work 
with  University  of 
Washington 
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1  n.  Include  the  capability  for  analyses  to  be  run  at  preset  periods  several  times  per  day  and 
on  demand  so  that  results  can  be  refreshed  if  data  are  received  from  hospitals  at 
different  times  throughout  the  day. 

np 

It.  Include  the  ability  to  perform  data  extracts  for  research  (with  written  approval  of  each  LHJ, 
as  some  existing  data  sharing  agreements  with  hospitals  may  explicitly  forbid  use  of 
data  for  non-Public  Health  purposes.) 

np 

1w.  Include  the  ability  to  handle  different  identifiers  and  de-identifiers,  with  a  goal  towards 
standardized  de-identifiers  and  identifiers. 

present 

1  r.  Provide  the  hospitals  with  the  same  analysis  capability  for  their  data  as  LH  Js  and  DOH 
have. 

present 

2e.  Communicable  Disease  case  reports 

np 

2f.  Primary  care  discharge  information 

np 

2i.  PHIMS  reports 

np 

2j.  School  and  business  absenteeism  data 

np 

2k.  Medical  examiner  data 

np 

21.  EMS/911  dispatch  data 

np 

2a.  Poison  Control  Data 

np 

1i.  Include  the  complete  ESSENCE  analysis  and  reporting  functionality  as  a  part  of  the  ODIN 
system. 

np 

Anticipated  but  no 
formal 

requirements 

created 


Anticipated  but  no 
formal 

requirements 

created 


Anticipated  but  no 
formal 

requirements 

created 


Anticipated  but  no 
formal 

requirements 

created 


Not  anticipated 


Anticipated  but  no 
formal 

requirements 

created 


Anticipated  but  no 
formal 

requirements 

created 


Anticipated  but  no 
formal 

requirements 

created 


Anticipated  but  no 
formal 

requirements 

created 


Requirement 
infeasible  due  to 
factors  outside 
developers 
control 
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APPENDIX  D 


Puerto  Rico  Conference  Poster 
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Puget  Sound  Infectious  Disease  Tracking  System 


iCIRG 


OF- 


:  a" 


Foundation  for 
Health  Care 
Quality 


data 


71  F 


& 


i)  Clinical  Informatics 
Research  Group 


Outbreak  Detection  Information  Network 


18 


Peter  J  Dunbar,  MD,  ChB,  MBA,  Ian  S  Painter,  PhD.,  Linda  Lekness  RN,  MBA 

Foundation  for  Health  Care  Quality,  705  2nd  Ave,  Suite  703,  Seattle  WA  98104 


Policy  development 


Technical  system  development 


Introduction 


We  have  identified  several  issues  that  are  potential  barriers  to  examining  and 
developing  the  necessary  policies  for  a  shared  data  system,  including: 

■jurisdictional  issues  between  the  local  public  health  agencies  and  the  state 
public  health  agency 

■questions  about  the  cost-benefit  equation  for  syndromic  surveillance 

■overlap  with  other  projects 

■data  sharing  and  privacy  concerns 

■organizational  cultural  differences  between  public  health  agencies  and  other 
groups  involved  in  the  process 


The  purpose  of  this  project  is  to  investigate  the  feasibility  of  implementing  a 
cross-jurisdictional  disease  surveillance  system  in  the  Puget  Sound  region. 
The  major  goals  of  this  project  are: 

■To  examine  the  technical  feasibility  of  setting  up  a  shared  data-analytic  appli¬ 
cation  across  a  number  of  jurisdictions 

■To  understand  the  relative  utility  of  data  gathered  through  these  techniques 
for  all  the  involved  parties 

■To  identify  and  develop  technical  and  policy  approaches  to  cooperation  in¬ 
cluding  the  implication  of  bi-directional  data  exchange  between  civilian  and 
military  public  health  surveillance  systems. 
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After  exploring  several  options,  the  Foundation  for  Health  Care  Quality  has 
engaged  several  local  health  jurisdictions  (Seattle  King  County  Public  Health 
Kitsap  County  Health  District,  Tacoma-Pierce  County  Health  Department  and 
Snohomish  Health  District)  and  the  Washington  State  Department  of  Health  in 
a  process  to  develop  a  charter  that  will  be  used  to  address  policy  issues  that 
arise  from  sharing  surveillance  data  across  jurisdictional  lines. 
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To  answer  these  questions  the  project  has  been  split  into  three  parts: 

-  Policy  development  process 

-  Technical  system  development 

-  Ongoing  research  capability  development 
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System  development  was  contracted  to  Paladin  Data  Systems,  a  data  base  and  application  development  company.  Technical  aspects  of  the  develop¬ 
ment  have  proceeded  very  smoothly,  the  use  of  a  professional  software  development  organization  has  proven  to  be  a  major  factor  in  the  progress  to 
date. 


The  long  term  goal  of  this  process  is  to  develop  a  process  that  can  be  used  to 
address  policy  issues  for  any  public  health  cross  jurisdictional  information 
technology  project. 


A  major  issue  has  been  the  difficulty  in  engaging  the  local  public  health  agencies  directly  into  the  development  process.  This  issue  has  been  ad¬ 
dressed  in  the  development  process  as  follows: 

■By  using  a  flexible  application  and  data  base  architecture 

■By  making  use  of  a  surrogate  end  user  with  domain  expertise  to  help  develop  detailed  requirements. 

■By  using  the  Agile  development  process,  with  one  month  development  cycles 

■By  development  of  a  prototype  which  is  being  used  to  refine  the  user  interface  and  optimize  application  workflow. 

■Through  use  of  a  large  and  realistic  simulated  data  set,  which  simulates  all  ER  visits  in  the  State  of  Washington  over  a  two  year  period,  and  includes 
a  simulated  major  disease  outbreak  and  simulated  data  errors. 


Results  and  Findings 


-The  technological  challenges  were  easily  addressed  by  the  contractor. 

A  prototype  application  (called  ODIN)  has  been  developed.  It  is  being  used  to 
demonstrate  the  technology  and  as  a  tool  to  elicit  more  detailed  requirements. 
Design  of  version  II  is  underway. 


Ongoing  research 


The  University  of  Washington’s  Center  for  Public  Health  Informatics  was  con¬ 
tracted  to  develop  methods  for  public  health  informatics  in  areas  of  use  to 
syndromic  surveillance.  Among  other  research  activities,  the  Center  is  devel¬ 
oping  an  online  application  to  allow  public  health  agencies  to  analyze  the  per¬ 
formance  of  standard  syndromic  surveillance  algorithms  on  their  own  data,  us¬ 
ing  an  internet  services  model. 


-Substantial  challenges  exist  in  integrating  the  local  health  jurisdictions 
(county  public  health  departments)  and  state  department  of  health  into  the 
project.  Our  initial  approach  to  engage  these  parties  did  not  meet  with  suc¬ 
cess.  A  revised  approach  is  proving  more  successful.  Publication  of  the  meth¬ 
ods  used  in  this  approach  will  be  beneficial  to  other  parties  outside  of  the  pub¬ 
lic  health  agencies  which  need  to  work  with  public  health  agencies  on  infor¬ 
matics  projects. 


Version  2  page  layouts  (under  development) 


^Shoki  client  -  Microsoft  Internet  Explorer 


=JSJXJ 


Address  |  washington.edu/OdinClient/Login5uccess.do  7]  jfl  Go 

- - - 


O-.  '*]  ,~l 


File  Edit  View  Favorites  Tools  Help 


tr 


3 


Shoki  Client 


r?~»i  .1  I  v  F  a  t  I  T  T  o  f 

WASHINGTON 


Main -Outbreak  Detection  Information  Network- Microsoft  Internet  Explorer 


CIRG 


Main -Outbreak  Detection  Information  Network- Microsoft  Internet  Explorer 


ODIiJ> 


ODI  i? 


Contacl  Us  i  Help  i  Change  Password  i  Logoff 


Contact  Us  i  Help  i  Change  Password  i  Logoff 


Main 


Main 


Clinical  Informatics 
Research  Group 


Status:  Ready 


CMbnul  Octectiir  Wornstli*' 


OijMwl  OOMilce  Intom 


Oaift.ro 


Welcome  Dr.  John  Snow  (London) 
Today  is  Thursday.  April  27, 2006 


Welcome  Dr.  John  Snow  (London) 
Today  is  Friday,  April  28,  2006 


r 


das 


1/ 


£ 


tasks 


l 


dashboard 


iiireiftl 


Data  Sets  |  Classification  1 1  Detection  1 1  Visualization  1 


Critical  Alerts  ! 


6 


Data  Quality 


Daily  Tasks 


All  Tasks -Including  Minutae 


Follow-up  Items 


J  Visits  [I  Visits  ->  Counts  [J  Counts  [j  Simulation  [J  Import  [ 
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. Original  Message . 

From:  Andrew  Fallat  [mailto:afallat@qualityhealth.org] 

Sent:  Tuesday,  December  23,  2003  2:34  PM 

To:  'Ward,  Sherry  L  Dr  USAMRMC' 

Cc:  'Peter  Dunbar  (E-mail)';  'Todd  Langton  (E-mail)’ 

Subject:  PR033147  -  Puget  Sound  Infectious  Disease  Tacking  System  (PSIDTS) 

The  purpose  of  this  email  is  to  follow-up  on  our  telephone  conversation  on  Monday,  December  22,  2003.  We 
appreciate  your  observations  and  believe  we  have  addressed  each  topic.  I  am  concerned  that  the  memo  is  longer 
than  anticipated  but  I  concluded  that  it  was  better  to  provide  a  more  thorough  record  for  your  review. 

Does  PSIDTS  still  have  value  to  DoD? 

PSIDTS  has  immediate  and  time  sensitive  value  to  Department  of  Defense  (DoD)  personnel.  We  are  concerned 
that  benefit  may  be  compromised  if  the  Proposal  is  delayed  and  opportunity  to  work  with  DoD  ESSENCE 
leadership  at  Madigan  Army  Medical  Center  is  lost  during  DoD's  JSIPP  implementation.  We  believe  that  the 
(unfunded)  work  we  have  accomplished  since  receiving  Patricia  Evans,  Contracting/Grants  Officer's  9/10/03 
Recommendation  for  Funding  letter  has  significantly  improved  the  Proposal's  value  to  the  military. 

The  Military  Relevance  Statement  provided  with  the  Proposal  remains  sound,  without  any  reservation 
whatsoever.  PSIDTS  will  develop  and  evaluate  different  techniques  to  acquire  data  on  civilian  health  populations. 
In  collaboration  with  cross-jurisdictional  public  health  leaders,  this  data  provides  the  first  Immediate  Benefit:  a 
regional  sentinel  system.  The  Long  Term  benefits  remain  the  same,  with  the  exception  of  researching  additional 
means  of  integrating  with  ESSENCE  rather  than  a  web-based  system.  This  improvement  addresses  a  weakness 
identified  and  articulated  by  DoD  Programmatic  Reviewers  in  their  critique  (see  next  section). 

The  9/10/03  Funding  letter  recommended  that  a  partnership  with  researchers  at  the  Department  of  Defense 
Global  Emerging  Infections  System  http://www.geis.ha.osd.mil/.  We  have  accomplished  this  partnership  and 
have  secured  GEIS  participation  on  PSIDTS  Steering  Committee,  through  LTC  Julie  Pavlin,  Head,  Electronic 
Surveillance  System  for  the  Early  Notification  of  Community-Based  Epidemics  (ESSENCE) 
http://www.geis.ha.osd.mil/GEIS/SurveillanceActivities/ESSENCE/ESSENCE.asp. 

Through  collaboration  with  Madigan  Army  Medical  Center  and  ESSENCE  we  learned  that  Fort  Lewis  (the  home 
base  for  Madigan  Army  Medical  Center)  was  selected  as  a  JSIPP  site.  PSIDTS  is  committed  to  partnership  and 
value  to  DoD:  therefore  it  appeared  to  increase  PSIDTS's  value  to  DoD  to  work  within  the  priorities  and  interests 
of  DoD  as  established  for  Fort  Lewis. 

http://www.geis.ha.osd.mil/GEIS/SurveillanceActivities/ESSENCE/JSIPPexsum.asp. 

Our  partnership  with  GEIS,  ESSENCE  and  Madigan  has  increased  value  to  DoD.  On  the  other  hand,  if  we  fail  to 
move  ahead  and  secure  the  immediate  benefits  of  a  regional  sentinel  system  in  the  strategically  valuable  Pacific 
Northwest  (using  the  collaborative  interests  of  DoD's  other  major  initiative  ESSENCE)  we  may  lose  a  time 
sensitive  opportunity. 


Does  the  absence  of  a  letter  confirming  access  to  data  on  military  subjects  merit  a  delay? 

The  absence  of  a  letter  confirming  access  to  data  on  military  subjects  is  not  unusual  at  this  stage  in  the  Proposal. 
In  fact,  it  was  anticipated  that  Tasks  1  and  3  would  articulate  the  privacy/security  issues  prior  to  securing  access. 
It  may  be  ironic  that  a  Proposal  designed  to  research  practices/policies  to  improve  partnership  between 
civilian/DoD  may  be  delayed  because  it  could  not  secure,  in  advance,  an  objective  that  was  scheduled  for  its  end. 

Unlike  research  utilizing  the  clinical  details  of  specific  DoD  personnel  (and  therefore  clearly  described  in  a 
proposal  and  dependent  on  access  to  clinical  data  on  military  subjects),  PSIDTS  relies  on  access  to  civilian  data 
which  creates  a  regional,  sentinel  system  available  to  DoD.  This  civilian  information  is  valued  by  DoD,  will  be 
integrated  into  ESSENCE  and  used  by  DoD  to  expand  its  knowledge  of  Community-based  Infections  affecting 
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DoD  personnel.  It  is  the  intent  and  on-going  practice  of  DoD/ESSENCE  to  share  data  with  civilian  authorities; 
however,  before  a  specific  confirmation  of  access  can  be  solicited,  Proposal's  Task  1  needs  to  begin. 

Access  to  data  on  military  subjects  is  being  addressed  in  accordance  with  the  original  Statement  of  Work.  DoD 
leadership  accepts  responsibility  for  releasing  access  to  data  and  will  do  so  only  when  the  utility  to  DoD  is 
addressed  and  security  in  place  to  assure  mandated  confidentiality.  Indeed,  this  work  was  anticipated  in 
Proposal's  Tasks  1c,  Id,  3a,  3c,  3d  and  3e  (all  related  to  developing  standards  and  policies  related  to  data 
access). 

It  was  also  recognized  that  PSIDTS  may  be  in  partnership  with  either  Madigan  or  GEIS,  or  both.  (Tasks  3a,  3d) 
We  secured  a  Letter  of  Collaboration  with  Madigan  indicating  that  more  specific  requests  would  go  to  their  IRB  if 
and  when  that  request  was  essential.  We  consulted  with  LTC  Julie  Pavlin  who  described  the  ESSENCE 
approach  to  access  to  data  on  military  subjects  (see  attached  Word  document  entitled  "Memo  for  Sharing),  in 
which  citations  are  provided  indicating  that  it  is  both  authorized  and  previous  practice  for  DoD  to  share 
surveillance  data  with  civilian  authorities. 

B 

Memo  for  sharing 
TMA  data.doc 

We  conclude  that  a  major  accomplishment  of  this  Proposal  will  be  to  assist  civilian  communities  and  DoD  leam 
methods  and  policies  essential  to  timely  and  appropriate  sharing  of  surveillance  information.  We  believe  that 
DoD's  ESSENCE  project  is  leading  the  way  and  we  intend  to  work  with  ESSENCE  through  Task  1  to  bring  in 
other  military  approvals  as  necessary. 

We  conclude  it  is  futile  (and  perhaps  a  detriment  to  developing  future  infectious  surveillance  partnerships  with 
DoD)  to  attempt  to  secure  detailed  letters  of  assurance  without  first  accomplishing  the  work  described  and  funded 
in  Task  1. 


Should  the  Statement  of  Work  be  amended  and  resubmitted? 

The  Statement  of  Work  remains  an  accurate  description  of  our  Proposal.  We  will  broaden  input  methods  into  the 
Surveillance  system,  in  recognition  of  comments  made  by  Programmatic  Reviewers  and  other  DoD  leaders  in 
ESSENCE. 

The  Statement  of  Work  describes  five  major  Tasks,  with  16  sub-tasks.  Task  2  identifies  developing  a  web-based 
case  reporting  system.  The  four  Task  2  sub-tasks  are  generic  to  case  reporting  and  not  dependent  on  web-based 
reporting.  Collaboration  with  DoD  personnel  within  ESSENCE  indicates  that  other  input  systems  are  preferred. 
These  comments  are  consistent  with  weaknesses  identified  in  the  Programmatic  Reviewer's  comments  (p.7): 

"The  proposal  may  overestimate  the  workability  of  the  Web-based  tool  for  providers  to  use".  And  again:  "One 
concern  relates  to... the  time  required  to  input  data  into  the  automated  data  collection  system".  The  four  sub-tasks 
under  Task  2  remain  relevant  except  that  they  will  be  applied  to  other  collection  methods  consistent  with 
ESSENCE.  Stated  differently,  all  16  of  the  sub-tasks  are  relevant  and  will  be  performed. 

We  also  believe  that  bi-directional  exchange  of  information  will  occur  with  DoD,  in  accordance  with  the  stated  goal 
of  ESSENCE  and  the  established  practice  of  DoD  related  to  Infectious  Surveillance.  We  see  no  change  to 
Statement  of  Work  in  that  arena. 

I  have  neither  expertise  nor  experience  on  which  to  base  a  conclusion  regarding  the  necessity  of  an  amendment. 

It  seems  to  be  administratively  complex,  however,  to  resubmit  a  new  Statement  of  Work  because  of  a  heading 
change  when  all  the  sub-tasks  remain  the  same.  We  believe  the  input  process  was  improved  by  replacing  a 
technique  that  was  described  as  a  "weakness"  by  Programmatic  Reviewers  with  another  process  preferred  by 
DoD  partners  we  were  encouraged  by  DoD  to  invite  to  the  Proposal. 


Is  "cost-reimbursed"  grant  acceptable  to  the  Foundation? 

Yes,  a  cost-reimbursement  grant  award  is  acceptable  to  the  Foundation.  The  Foundation  does  not  have  working 
capital  sufficient  to  make  other  than  nominal  advances  to  staff  and  vendors.  Likewise,  the  Foundation  cannot 
risk  incurring  expenses  without  assurance  that  they  will  be  reimbursed.  As  long  as  "cost-reimbursed"  permits 
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both  a  reasonable  amount  of  working  capital  and  staffing  adequate  to  handle  Foundation  requests  (with  quick  turn 
around)  for  assurance  about  specific  potential  expenses,  then  "cost-reimbursed"  is  acceptable. 


Thank  you  for  the  opportunity  to  respond  to  your  observations.  We  appreciate  your  consideration  of  funding 
decisions  essential  to  our  beginning  work  on  PSIDTS.  We  anticipate  your  call  to  Todd  Langton  on  December  30. 

Andrew  Fallat 
President/CEO 

Foundation  for  Health  Care  Quality 
Seattle,  WA 
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Agreement  between  FHCQ  and  DOH  -  to  insert 
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DOH  CONTRACT  NUMBER:  N15526 


AGREEMENT 

BETWEEN  FOUNDATION  FOR  HEALTH  CARE  QUALITY  (FHQ) 

AND 

STATE  OF  WASHINGTON  DEPARTMENT  OF  HEALTH  (DOII) 


PURPOSE:  The  purpose  of  this  contract  is  to  contrast  two  syndromic  surveillance  software  applications  in  their 
ability  to  gather  process  and  provide  disease  incidence  data  on  which  to  base  public  health  response  decisions. 

TERM:  Subject  to  its  other  provisions,  the  period  of  performance  under  this  contract  shall  be  from  December  1,  2006 
through  December  31, 2007  unless  sooner  terminated  as  provided  herein. 

STATEMENT  OF  WORK:  DOH  shall  provide  the  goods  and  services  as  described  in  EXHIBIT  A,  STATEMENT  OF 
WORK,  attached  hereto  and  incorporated  herein. 

CONSIDERATION:  The  maximum  consideration  available  under  this  contract  is  $  payable  in  accordance  with 

EXHIBIT  A,  attached  hereto  and  incorporated  herein. 

SOURCE  OF  FUNDS:  FEDERAL:  S  STATE:  $i  OTHER:  $0  TOTAL:  $. 

Federal  funds  disbursed  through  this  contract  were  received  by  FHQ  through  OMB  Catalogue  of  Federal  Domestic 
Assistance  Number:  12.420.  DOH  agrees  to  comply  with  applicable  rules  and  regulations  associated  with  these  federal 
funds. 

INVOICES  AND  PAYMENT:  Department  of  Health  will  bill  the  Foundation  for  Health  Care  Quality  on  a  quarterly  basis, 
and,  within  45  days  of  the  end  of  the  quarter.  The  invoice  for  the  last  quarter  of  2007  will  be  billed  no  later  than  December 
31,  and  will  be  due  to  Department  of  Health  by  January  15,  2008. 

FHQ  will  make  payment  to  DOH  within  30  days  upon  receipt  of  properly  executed  invoice  vouchers. 

PRECEDENCE:  In  the  event  of  conflict  or  inconsistency,  the  following  order  of  precedence  shall  apply:  (1)  federal  law 
and  regulation,  (2)  state  law  and  regulation,  (3)  this  agreement. 

UNDERSTANDING:  This  contract  including  referenced  exhibits,  attachments  and  documents  included  by  reference 
contains  all  the  terms  and  conditions  agreed  upon  by  the  parties.  No  other  understandings,  oral  or  otherwise,  regarding  the 
subject  matter  of  this  contract  shall  exist  or  bind  any  of  the  parties  hereto. 

The  parties  to  this  agreement  agree  to  abide  by  and  fully  comply  with  the  provisions  set  forth  herein. 
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GENERAL  CONDITIONS 


* 

1.  AMENDMENTS  -  This  contract  may  be  amended  by  mutual  written  agreement  of  the  parties.  Such  amendments  shall 
not  be  binding  unless  they  are  in  writing  and  signed  by  personnel  authorized  to  bind  each  of  the  parties. 

2.  CONFIDENTIALITY/SAFEGUARDING  OF  INFORMATION  AND  PRIVACY  -  The  use  or  disclosure  by  any 
party  of  any  information  concerning  a  client  obtained  in  providing  service  under  this  agreement  shall  be  subject  to 
Chapter  42.56  RCW  and  Chapter  70.02  RCW,  as  well  as  any  other  applicable  federal  and  state  statutes  and  regulations. 
Personal  information  collected,  used  or  acquired  in  connection  with  this  contract  shall  be  used  solely  for  the  purposes  of 
this  contract  and  other  public  purposes. 

3.  FEDERAL  CERTIFICATIONS  AND  ASSURANCES  AND  DEBARMENT  AND  SUSPENSION  -  FHQ  and  DOH 
agree,  that  all  activities  pursuant  to  this  Agreement  will  be  in  accordance  with  applicable  current  or  future  federal,  state, 
and  local  laws,  rules,  and  regulations.  DOH  certifies  that  it  is  not  debarred,  suspended,  or  otherwise  excluded  from  or 
ineligible  for,  participation  in  Federal  or  State  government  contracts.  DOH  further  certifies  that  it  will  not  contract  with 
a  Subcontractor  that  is  so  debarred  or  suspended. 

4.  DISPUTES  -  The  parties  shall  use  their  best,  good  faith  efforts  to  cooperatively  resolve  disputes  and  problems  that  arise 
in  connection  with  this  Contract.  Both  parties  will  continue  without  delay  to  carry  out  their  respective  responsibilities 
under  this  contract  while  attempting  to  resolve  the  dispute  under  this  section.  When  a  genuine  dispute  arises  between 
die  FHQ  and  DOH  regarding  the  terms  of  this  agreement  or  the  responsibilities  imposed  herein  which  cannot  be  resolved 
at  the  project  management  level,  either  party  may  submit  a  request  for  a  dispute  resolution. 

This  dispute  resolution  process  constitutes  the  sole  administrative  remedy  available  under  this  contract.  The  parties 
agree  that  this  resolution  process  shall  precede  any  action  in  a  judicial  and  quasi-judicial  tribunal. 

5.  GOVERNING  LAW  -  This  contract  shall  be  governed  by  the  laws  of  the  State  of  Washington  and  applicable  federal 
laws  and  regulations.  The  Venue  of  any  legal  action  or  suit  concerning  this  agreement  shall  be  the  Thurston  County 
Superior  Court  and  all  actions  or  suits  thereon  shall  be  brought  therein. 

6.  HOLD  HARMLESS  -  Each  party  shall  defend,  protect,  and  hold  harmless  the  other  party  from  and  against  all  claims, 
suits,  and/or  actions  arising  from  and  negligent  or  intentional  act  or  omission  of  that  party’s  employees,  agents,  and/or 
authorized  subcontractors)  while  performing  this  contract.  Claims  shall  include,  but  not  be  limited  to,  assertions  that 
the  use  or  transfer  of  any  software,  book,  document,  report,  film,  tape,  or  sound  reproduction  or  material  of  any  kind, 
delivered  hereunder,  constitutes  an  infringement  of  any  copyright,  patent,  trademark,  trade  name,  or  otherwise  results  in 
an  unfair  trade  practice. 

7.  LIMITATION  OF  AUTHORITY  -  Alterations,  modifications,  or  waivers  of  any  clause  or  condition  of  this  contract  is 
effective  or  binding  only  if  in  writing  and  signed  by  duly  authorized  representatives  of  both  parties. 

8.  SUBCONTRACTING  -  DOH  is  responsible  for  ensuring  performance  of  its  subcontractors  and  that  all  terms, 
conditions,  assurances  and  certifications  set  forth  in  this  agreement  are  carried  forward  to  any  subcontracts. 

9.  TERMINATION  -  Except  as  otherwise  provided  in  this  contract,  either  party  may,  by  THIRTY  (30)  calendar  days 
written  notice,  beginning  on  the  second  day  after  the  mailing,  terminate  this  contract  in  whole  or  in  part  when  it  is  in  the 
best  interests  of  that  party. 
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APPENDIX  G 


HIPPA 

Status  of  Legal  Issues 


Analysis  and  recommendation  of  HIP  A  A  application  provided  by  John  R.  Christiansen,  Christiansen  IT 
Law ,  a  law  firm  specializing  in  technology  related  services 


Legal  Issues  Resolved  to  Date: 


Issue 

Discussion 

Scope  of  data 
available  for 
contribution  to 
ODIN 

HIPAA  prohibits  disclosure  of  Protected  Health  Information  for  most 
public  health  purposes,  unless  individual  authorization  is  obtained. 
Obtaining  such  authorization  may  be  difficult  or  sometimes 
impossible.  However,  HIPAA  expressly  does  permit  disclosure  of 
‘‘Limited  Data  Sets"  for  public  health  purposes  without  such 
authorization.  Limited  Data  Sets  appear  to  provide  sufficient 
information  for  ODIN  purposes. 

Authority  to  use 
data  processing 
vendors  to 
provide  ODIN 
data 

Many  hospitals  rely  upon  data  processing  services  vendors  to  store 
and  manage  their  Protected  Health  Information.  However,  HIPAA 
requires  Covered  Entities  (including  hospitals)  to  limit  the  uses  and 
disclosures  such  vendors  may  make  of  their  Protected  Health 
Information.  Model  contract  provisions  authorizing  vendors  to  create 
and  disclose  Limited  Data  Sets  on  behalf  of  hospitals  for  public  health 
purposes,  including  ODIN,  have  been  developed  and  found  acceptable 
to  both  a  major  data  processing  services  vendor  and  a  number  of 
hospitals. 

Legal  Issues  to  be  Resolved 


Issue 

Discussion 

Specify  public 
health  reporting 
laws  supporting 
disclosure  of 
ODIN  data 

HIPAA  authorizes  disclosure  of  Protected  Health  Information  for 
public  health  purposes  but  requires  that  those  exceptions  be  specified. 
Public  health  laws  requiring  or  permitting  disclosure  may  be  either 
federal  or  state.  There  may  be  a  number  of  alternative  public  health 
legal  bases  for  ODIN  disclosures;  the  specific  legal  bases  therefore 
should  be  specified. 

Specify  public 
health  authority 
of  ODIN  data 
receivers  to 
obtain  Protected 

HIPAA  requires  legal  authority  for  entities  to  receive  Protected 

Health  Information  from  Covered  Entities  for  public  health  (and 
other)  purposes.  Public  health  authority  is  usually  but  not  necessarily 
vested  in  governmental  authorities  (federal,  state  or  local).  The 
specific  authority  for  the  ODIN  data  receivers  to  obtain  such 
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Health 

Information 

information  should  be  specified,  or  if  none  currently  exists  should  be 
developed. 

Establish 
security 
governance  for 
ODIN 

participation 

In  order  to  provide  assurance  to  ODIN  participating  organizations 
that  they  will  be  able  to  maintain  HIPAA  security  compliance,  a 
security  governance  committee  should  be  established  which  develops 
policies,  procedures  and  technical  guidance  for  ODIN  participation. 

Publish  legal 
interpretation 
supporting  ODIN 
participation 

In  order  to  encourage  ODIN  participation,  a  legal  opinion  (perhaps  in 
the  form  of  a  white  paper)  should  be  published  which  summarizes  the 
legal  bases  for  participation  in  compliance  with  HIPAA  and 
Washington  state  law. 
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